[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Large repository crashing cvs2svn.py and possible memory leak?

From: Nathan Sharp <nsharp_at_phoenix-int.com>
Date: 2002-12-19 13:58:15 CET

I am about 2 days old to subversion, I wasn't even aware that there was
a branch with anything I might be interested in. As Donald said, yes, I
am using the trunk.
I experimented further (thanks to some help I got on the IRC channel)
and think I have a workaround now. After running passes 1-3 (and
forcibly preventing 4 from running), I took the cvs2svn-data.s-revs file
and chopped it into files with 20,000 lines each. By running the script
on pass 4 on each file in order, I seem to be able to run successfully,
which proves that the problem a) is a memory leak and b) was failing
because it ran my system out of memory. The only negative effect of
what I did is that right where I split the files (since I just did
exactly 20k line files and didn't manually split the files up at a
natural commit break) I will end up with an commit which is split in 2,
which is minor enough for me not to worry about it.
The general belief I heard on the IRC channel is that the memory leak is
probably in the swig bindings and not in the cvs2svn.py script itself.
I'd be happy to help in any way possible if someone wants to try and fix it.


Branko Čibej wrote:

>Which cvs2svn.py script are you using? The one from thr trunk, or the
>one from /branches/cvs2svn-mmacek?
>Nathan Sharp wrote:
>>I'm currently testing out subversion on a test box here (RedHat 8.0)
>>and have been very impressed so far. I'm quite excited to employ it
>>on a real project, but am having problems importing our large CVS
>>repository into it. Our CVS repository goes back to 1996 and is 2.2GB
>>on disk. The cvs2svn.py script successfully runs through the first
>>thee passes but fails with a segementation fault in the fourth pass.
>>The repository it generates is valid up until the point it crashes
>>(somewhere in 1999) and it doesn't seem to crash at the same spot if I
>>re-run it. One thing I noticed was that as the script runs, it takes
>>increasingly more and more memory as it goes, was up to almost 400Meg
>>last I checked before it crashed. I'm suspicious that perhaps it just
>>runs the computer out of memory (it isn't a real powerful box, it is
>>just for testing), but I don't have any real evidence to that fact.
>>I'm running:
>>svn HEAD as of a couple of days ago
>>python 2.2.1-17 RPM
>>swig 1.3.16 from tarball
>>viewcvs HEAD as of a couple of days ago
>>Berkely db 4.0.14-14 RPM
>>Any advice for debugging this? I ran the script w/ a -v and the
>>reports it generates seem O.K. up until it crashes, at which point the
>>output ends abruptly mid-line w/ no other errors. The shell reports
>>the segmentation fault. No core file is generated.
>>Thanks again!
>> Nathan
>>To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
>>For additional commands, e-mail: dev-help@subversion.tigris.org

To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Thu Dec 19 14:29:42 2002

This is an archived mail posted to the Subversion Dev mailing list.