[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: svn list -R of medium-size repository takes 10 hours.

From: Jean-Marc Godbout <jmgodbout_at_gmail.com>
Date: 2005-08-28 21:12:31 CEST

Here is a patch I will submit to the list later on today, after a
friend of mine runs all the unit tests on it as well.

The patch does an extra propfind before doing the "big" propfind to
probe and see if the server supports the deadprop-count prop. I
realise that this is an extra round trip. Here's why.

We do not do an OPTIONS request at the beginning. This means that to
implement the OPTIONS solution I would have needed to make a new
OPTIONS request. This would have been an extra round trip as well. I
think that it's still a good solution, just better suited to an
independant patch.

I then wanted to simply add the deadprop-count prop to a propfind in
the initialization phase. Unfortunatly I didn't find any of those
propfinds that had both a svn_ra_session_t (to store the result) and
the properties at the same time.

So my compromise was to do an extra propfind, but it's very small as
it only request one prop.

One thing is that I didn't remove all the DeltaV props from an allprop
request on the server as suggested in the bug report. If I did that,
than any unpatched clients would fail when trying to access a patched
server. That should still be considered though as it would also speed
up generic dav clients.

With this patch we are talking about time reductions in the order of
50-75% (sometimes even better) and bandwidth reductions of the same
magnitude. This should make a big difference for heavy users of ls.
Given that we are doing an extra PROPFIND, there is still room for
improvement, but this falls more into the domain of issue 1161.

Thank you everyone for your suggestions and tips. They were very useful.

On 8/26/05, Jean-Marc Godbout <jmgodbout@gmail.com> wrote:
> There was in fact a test break. I fixed it and will include it in my next patch.
>
> In issue 2151 (http://subversion.tigris.org/issues/show_bug.cgi?id=2151):
> "The URIs listed in gather_propsets() are returned in
> the OPTIONS response. Since we always(?) issue an OPTIONS request, then we
> extract them at that point, recording the presence in the session state."
>
> Unfortunatly, from what I can see int the code we only issue an
> OPTIONS request on commit. Should we always do it? I will try to add
> it and see if there are any issues that pop up.
>
>
> On 8/26/05, Jean-Marc Godbout <jmgodbout@gmail.com> wrote:
> > Oh, I'm sorry, I just realised I had not run all the tests. Please
> > don't waste your time looking at this patch until I make sure it
> > doesn't break tests.
> >
> > On 8/26/05, Jean-Marc Godbout <jmgodbout@gmail.com> wrote:
> > > I have a simple patch that trims down time by about half in my tests.
> > >
> > > It's NOT complete, here's what works:
> > >
> > > 1. Patched client + Patched server == fast
> > > 2. Patched client + Unpatched server == broken
> > > 3. Unpatched client + Patched server == same speed as before
> > > 4. Dav client + Patched server == same speed as before
> > >
> > > I need to fix #2 before considering it a real patch.
> > >
> > > Essentially this implements some of the suggestions in 2151 - starting
> > > with Ben's patch. Note that I don't think it's wise to remove DeltaV
> > > props from an allprop PROPFIND as this would break unpatched clients
> > > trying to access patched servers. We can discuss this though.
> > >
> > > On 8/25/05, Carsten Koch <Carsten.Koch@icem.com> wrote:
> > > > Jean-Marc Godbout wrote:
> > > > > For Ben's suggestion of using mod_deflate, I think it could be useful
> > > > > in your case. I just ran a test on a real system and a sample svn list
> > > > > went from 60k to 16k.
> > > >
> > > > We have mod_deflate active now on our svn server and the small test case
> > > > (svn list -R on a directory tree with only 25 entries) goes down from
> > > > 8 seconds to 3 seconds.
> > > > Not the facto 100 improvement I am looking for, but nevertheless more
> > > > than twice as fast.
> > > >
> > > > Thanks to Ben and you for the suggestion!
> > > >
> > > > Carsten.
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> > > > For additional commands, e-mail: dev-help@subversion.tigris.org
> > > >
> > > >
> > > >
> > >
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> > For additional commands, e-mail: dev-help@subversion.tigris.org
> >
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Received on Sun Aug 28 21:14:00 2005

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.