[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Problem with large files

From: Erik Huelsmann <ehuels_at_gmail.com>
Date: 2006-08-28 16:14:26 CEST

On 8/28/06, Ben Collins-Sussman <sussman@red-bean.com> wrote:
> On 8/28/06, Daniel Berlin <dberlin@dberlin.org> wrote:
> > The report from the one person who has ever tried it with large files
> > was that it sped up commit times from 45 minutes to less than 5 ;)
> I don't think that this is rocket science which requires testing. :-)
> Of course, if you just insert new data directly into the stream
> without trying to deltify it, it's gonna be way way faster.

As long as the pipe it has to go through can offer the capacity...

> What's tricky here is coming up with a design. Should the svn client
> be magically deciding when to deltify or not, based on some heuristic?

Well, I can think of an algorithm, not a heuristic, but I think that
crosses every realistic boundary of layers: just send data until the
pipe fills up; take all the time it takes for the pipe to become
available to do delta-encoding, send (potentially encoded) data as
soon as the pipe becomes available.

This way, you'll get (roughly) maximum throughput on the network,
assuming 'just sending' outweighs encoding any time.

> Or should it be controlled by the user via switches or
> config-options? We have a really long standing issue filed (like...
> years old) about giving users the option to toggle compression on the
> fly (something akin to 'cvs -zN'). Is that the interface we want?

The open issue is for import only, but, why shouldn't we offer the
interface? I don't use cvs -z3 with all repositories, so, I use it as
a switch...



To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Mon Aug 28 16:16:53 2006

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.