[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Performance bug? Very large rev files

From: Rick Yorgason <rick_at_ldagames.com>
Date: 2007-03-30 09:11:40 CEST

Roy Franz wrote:
> Are the delta's just processed by the server or also by the client?
> If only the server
> uses these deltas (and therefore cares about the window size) then
> this could be handled by just bumping the repository format, right?
> The gains on larger binary files seems like it could be very
> significant. In my testing, I saw xdelta producing deltas several
> times smaller than the corresponding subversion rev files. (I believe
> that I was properly accounting for the skip-revs.)

Well, I haven't peeked at too much of the code, and I'm a noob on this
list, so forgive me if I'm making an idiot of myself, but I'm pretty
sure I can answer these:

1) The client also has to produce deltas. Otherwise it would be
transmitting the entire file when a small change is made. But it's moot
anyway because...

2) Simply bumping the repository format, without some way to convert
between them, wouldn't work. Otherwise, what would you send to a client
that doesn't understand svndiff2 deltas? (I'm fairly sure Subversion
already translates between svndiff0 and svndiff1 for
backwards-compatibility, but all that has to deal with is zlib
(de)compression.)

That said, only the server would need to do any conversions, because the
client and server can negotiate which formats they understand, so you
have two new backwards-compatibility profiles:

When client = svndiff2, server = svndiff1 (the simple one):
Client uses svndiff1 instead of svndiff2 in all transactions.

When client = svndiff1, server = svndiff2:
Server degrades svndiff2 deltas into svndiff1 deltas before
transmitting. Client sends svndiff1 deltas to server, server upgrades
them, or uses them directly (if the server supports variable window
sizes, it would work, but the clients could affect the long-term
performance of the server; granted it would be no worse than it
currently is).

Unless I'm missing something, translating between svndiff1 and svndiff2
could be done in a relatively safe and straight-forward way, as long as
we only allowed the window size to be multiples of 100k, but it would
hog some extra resources. To degrade from (say) a 1000k window to a
100k window, you simple reconstruct that window and then deconstruct it
into 10 100k windows, and vice versa for upgrading deltas.

I suppose if people wanted to support pre-svndiff2 clients and didn't
want to suffer the extra ram/cpu cycles on their server they could
specify --pre-(version-with-svndiff2)-compatible, or, if the server can
specify a maximum window size, just set it to 100k.

-Rick-

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Fri Mar 30 09:11:33 2007

This is an archived mail posted to the Subversion Dev mailing list.