[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Performance bug? Very large rev files

From: Daniel Berlin <dberlin_at_dberlin.org>
Date: 2007-03-30 04:25:54 CEST

On 3/29/07, Rick Yorgason <rick@ldagames.com> wrote:
> Daniel Berlin wrote:
> > On 3/29/07, Malcolm Rowe <malcolm-svn-dev@farside.org.uk> wrote:
> >> I'm not sure whether this is the cause, but be aware that we compute
> >> the deltas using a window of 100K that moves (in 100K steps) through the
> >> reference and version files, so we can't take advantage of inter-window
> >> deltas. We do this to bound the memory usage of the compressor and
> >> decompressor, and it seems to be one of the most significant factors in
> >> our compression performance, especially if we ever want to start using
> >> some of the (theoretically) better delta algorithms.
> >
> > Ya, we need to up the window size (to about a meg or two), but it's
> > hardcoded in some places that make it a not backwards compatible
> > change with more work.
>
> Yep, that's definitely the, problem. To be fair, Subversion actually
> out-performs xdelta3 with a window that small, my a margin of about 5M.
>
> On the xdelta.org site, they imply that large files use a window of
> 100Mb, which roughly adds up (my original tests used a total of 6
> windows for the whole comparison, while the test I did today used a
> total of around 450 windows).
>
> Hmm, I wish I had time to pick up the Subversion source and see if I can
> fix this, but I have too many other priorities. Maybe in a couple
> months I'll see what I can do, if somebody else isn't already on it.
>

You only need to change one define, but it makes it incompatible with
subversions not compiled with that value :)

> -Rick-
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Fri Mar 30 04:26:10 2007

This is an archived mail posted to the Subversion Dev mailing list.