[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Speeding up cvs2svn (was Re: cvs2svn takes very long time to execute (days!))

From: <kfogel_at_collab.net>
Date: 2004-02-15 21:56:35 CET

Tobias Ringström <tobias@ringstrom.mine.nu> writes:
> Interesting, but for 1-3, it would be a lot faster to avoid the DB lib
> completely and use in-memory python hashes. One of the things that
> take a long time is the marshalling done when passing data to and from
> the DB. I have some code to do this, and I hope to commit it to trunk
> in a couple of days at the latest.

Oh-ho! The marshalling is more time-consuming than I thought, then.
(I hadn't profiled it yet, since correctness issues are a higher
priority to me right now, but still I admit this is surprising.)

A thought: we could cut down the marshalling quite a bit, by making
marshal/unmarshal behavior an optional flag to the Database class, and
passing it as false for those database which use only Python strings
as keys/values. The Database could still test keys/values for sanity
before using them, assuming type tests are still cheap (!).

I feel funny about using in-memory hashes. cvs2svn.py should scale
well by default. Do you plan to automagically switch to a disk
database if the hash count exceeds a certain magic number?

> Please note that cvs2svn is not yet 1.0 material, and it still has
> correctness bugs, i.e. the content of tags and branches can be
> incorrect. I'm about to commit a new tool that looks for such errors.

You, sir, rock my socks.


To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Sun Feb 15 22:58:25 2004

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.