[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Efficiency of large file import?

From: Marcin Kasperski <Marcin.Kasperski_at_softax.com.pl>
Date: 2004-03-25 01:24:45 CET

On Wed, 2004-03-24 at 16:30, Ben Collins-Sussman wrote:
> On Wed, 2004-03-24 at 06:42, Marcin Kasperski wrote:
> > I just tried importing large (~200MB) file into subversion.
> >
> > Good news: repository size =~ file size + constant overhead.
> >
> > Bad news: import time (time svn commit executed) > 1 hour
> > [ no net overhead - I experimented using file:// repos url]
> >
> > Are there any ideas of how to lower the import time
>
> Get a faster CPU or disk? :-)
>
> Seriously, 'svn import' isn't doing the same thing as 'cvs import'.
> It's not just blindly copying data into an RCS file. It's compressing
> the entire file using a binary-diff algorithm, and then stuffing the
> data into database tables. It's just slower.
>
> Someday SVN users might be able to turn off compression, but there's
> nothing you can do about it for now.

Hmm, on the same - in fact not too fast - machine:
- copying this file between two disks - about 1 minute
- bzipping it - a few minutes
subversion import still is more than order of magnitude slower...
So from my naive point of view it seems there is a room for improvement.

Of course it is likely that handling large files is not the highest
priority problem....

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Received on Thu Mar 25 01:25:13 2004

This is an archived mail posted to the Subversion Users mailing list.