[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Performance bug? Very large rev files

From: Malcolm Rowe <malcolm-svn-dev_at_farside.org.uk>
Date: 2007-03-29 10:30:59 CEST

On Thu, Mar 29, 2007 at 04:04:16AM -0400, Rick Yorgason wrote:
> For the last little while I've been diagnosing some very large rev files
> for all of the commits of one of my files in a real-world repository,
> and the db/rev files seem to be much bigger than they should be.
>
> I've posted some experiments in the users list, but I was hoping I could
> get some attention from the devs. My latest post shows just how big
> this issue can be:
>
> http://subversion.tigris.org/servlets/ReadMsg?list=users&msgNo=64261
>

I'm not sure whether this is the cause, but be aware that we compute
the deltas using a window of 100K that moves (in 100K steps) through the
reference and version files, so we can't take advantage of inter-window
deltas. We do this to bound the memory usage of the compressor and
decompressor, and it seems to be one of the most significant factors in
our compression performance, especially if we ever want to start using
some of the (theoretically) better delta algorithms.

You also didn't confirm whether the deltas are svndiff0 or svndiff1,
which would make an enormous difference if your data is self-compressible
- assuming you created the repository with Subversion 1.4.x (and didn't
specify the --pre-1.4-compatible flag), the db/format file should contain
the value '2', meaning that svndiff1 deltas can be used.

Regards,
Malcolm

  • application/pgp-signature attachment: stored
Received on Thu Mar 29 10:31:23 2007

This is an archived mail posted to the Subversion Dev mailing list.