[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: svn list -R of medium-size repository takes 10 hours.

From: Jean-Marc Godbout <jmgodbout_at_gmail.com>
Date: 2005-08-21 11:25:57 CEST

For Ben's suggestion of using mod_deflate, I think it could be useful
in your case. I just ran a test on a real system and a sample svn list
went from 60k to 16k.

To enable it, get mod_deflate compiled and installed, add the proper
LoadModule to your apache conf file. Then add "SetOutputFilter
DEFLATE" to your <Location ...> for the repository.

I think it would help you because of your relatively slow connection.
A word of caution though, in my case it was actually slower because of
the extra processing required by mod_deflate. In local networks I
suspect that mod_deflate would actually not improve performance at
all.

Do try it though and tell me if it makes a difference.

Jean-Marc

On 8/21/05, Jean-Marc Godbout <jmgodbout@gmail.com> wrote:
> I am currently looking into this issue. It seems like there is
> somewhat of an inherent slowness with WebDAV/DeltaV because of the
> amount of requests and the format in which they return data. The
> gigabytes of data do seem odd though.
>
> I think that it will always be slow to a certain extent, but there
> should be ways to improve it. One current issue is that mod_dav_svn
> (the apache module part of the subversion server). Is taking it's
> sweet time to answer certain requests that involve many different
> elements.
>
> Here is a simple example of a dav request - note that this is running
> in a debugger, in a VM so the times are simply examples:
>
> 1. There are a couple requests going to the server to resolve the
> proper folder (5 seconds of chatter).
> 2. Then the "proper" dav request is made that will list a folder.
> (mod_dav_svn takes 10 seconds to reply).
> 3. The folder listing is sent back to the client. (23 seconds of
> active network traffic).
>
> The parts that could be reasonably cut down would be 1 and 2. There is
> unfortunatly no simple solution for #3. There is alot of data being
> sent back. In your specific case given that you are effectively doing
> a ton of lists (-R), the amount of data likely comes from #3. If you
> could send me an ethereal log (assuming your stuff isn't too private)
> I could confirm this for you.
>
> Regards,
>
> Jean-Marc
>
> On 8/20/05, Tobias Ringström <tobias@ringstrom.mine.nu> wrote:
> > Ben Collins-Sussman wrote:
> >
> > > This is one of many situations where "being a DAV client" is hurting
> > > the svn client's performance. In DAV-land, the proper way to get a
> > > list of children in a collection (directory) is to do a depth-1
> > > PROPFIND. This has the unfortunate side effect of not just listing
> > > all children, but also all of their properties -- both user-generated
> > > and server-generated ones. All in XML.
> > >
> > > If we were to just say "forget it, the svn client isn't a DAV client"
> > > (which has been discussed before), we could make up yet another
> > > custom protocol response/request that only svn and mod_dav_svn would
> > > understand. We're already doing that in many places where subversion
> > > concepts don't line up with WebDAV/DeltaV. Doing it for the sake of
> > > performance gains would be a new policy. I don't think everyone is
> > > (yet) in agreement yet about this.
> >
> > Anyone who's interested in making 'svn ls' faster over http should take
> > a look at issue 2151.
> >
> > http://subversion.tigris.org/issues/show_bug.cgi?id=2151
> >
> > In this case, I'd say it's more a problem of how we use DAV, than a
> > problem with DAV itself.
> >
> > /Tobias
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> > For additional commands, e-mail: dev-help@subversion.tigris.org
> >
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Sun Aug 21 13:13:24 2005

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.