On Jan 22, 2008 9:07 AM, Branko Èibej <brane_at_xbc.nu> wrote:
>
> David Glasser wrote:
> > 2008/1/22 Niels Werensteijn <n.werensteijn_at_student.utwente.nl>:
> >
> >> Summary so far:
> >>
> >> New files, or new chunks of files have no source stream (chunk) to
> >> compare it to, in order to make a diff of it. At this moment, vdelta
> >> routine is used on these streams. This takes a lot of cpu power. Even
> >> worse, since the diffs are compressed with zlib later on (both on disk
> >> and during transmission), vdelta is actualy making it harder for zlib to
> >> compress the stream, resulting in larger streams.
> >>
> >> This patch:
> >> In this patch I replace the call to vdelta with a single call to
> >> svn_txdelta__insert_op and just insert the whole chunk as new data. This
> >> costs almost no cpu time, and lets zlib compress it better.
> >>
> >> My test results on two repositories confirmed that the on disk size
> >> shrunk a little bit, and that cpu time was greatly reduced.
> >>
> >> My time results:
> >> Export Repository1 (delphi source code): from 58,6 to 42,0 seconds
> >> Export Repository2 (big compressed bin): from 10,71 to 5,47 seconds
> >>
> >
> > This seems like a reasonable idea, but note that we don't always send
> > txdeltas compressed. Specifically, only the serialization of txdeltas
> > called "svndiff1" is compressed, not "svndiff0". Some examples of
> > when each can be used:
> >
> > * In FSFS repositories, svndiff1 is only used in repositories of DB
> > format 2 and above.
> >
> > * In the svnserve protocol, svndiff1 is only used if the client and
> > server both support it (and declare so in their header).
> >
> > I'm not sure what controls the use of svndiff1 in the DAV protocol and
> > BDB repositories off the top of my head.
> >
> > My guess (not based on actual experimentation) is that your patch
> > would be a performance regression for cases that svndiff0 is
> > transmitted. If that's the case (can you test it? One way is to
> > create an FSFS repository with --pre-1.4.x-compatible and compare
> > sizes), we'd probably want to rev a bunch of functions to add a
> > boolean flag "output_will_be_compressed" or something which gets
> > passed all the way down to your one-line change.
> >
>
> Bah. --pre-1.4.x-compatible is for old clients accessing via file://.
> svndiff0 over the wire is for old clients, period. I think burning that
> bit all the way through our APIs is more hassle than upgrading clients.
I'll buy that for the FS; although it's also for "old repositories who
haven't done a dump and load, the only officially supported way of
upgrading repository format", I suspect that 1.5 will require an
"svnadmin upgrade" command anyway.
Do we have actual numbers on client versions anywhere? It wasn't long
ago that it was uncommon to find 1.4 in stable versions of various
distributions.
--dave
--
David Glasser | glasser@davidglasser.net | http://www.davidglasser.net/
Received on 2008-01-24 01:22:27 CET