On Tuesday, June 18, 2002, at 11:23 AM, Brandon Ehle wrote:
> Daniel Berlin wrote:
>
>>>> Support for branches.
>>>>
>>>> I've converted *huge* repositories using the existing cvs2svn with no trouble.
>>>>
>>>> You need to grab rcsparse from viewcvs.
>>>>
>>>>
>>>>
>>> Ok, I've got a 20 gig repository CVS report with some branches. Is subversion going to be able to handle this large of a dataset? (Some of the individual files will exceed 100 megs as well.)
>>>
>>
>> The individual *RCS* files, or the individual files themeslves (IE they are 100 meg, checked out)?
>>
> We have some of both varieties, the largest RCS file for the 100 meg checked out ones is about 552megs.
>
Well, okay, here's your problems, in order
1. You need to recompile APR, APR-UTIL, and Subversion with largefile support (There's a patch for APR to do this somewhere), assuming you are on linux. Otherwise, you'll hit the 2 gig limit. It's just recompiling with -D_FILE_OFFSET_BITS=64 or something like that. It automatically does the right thing so you don't need to rename functions on your own or anything.
2. The stream window size is set to 100k. (subversion/include/svn_types.h, the define at the end of the file). Files over this length won't be deltified right now (until we have a delta combiner. See issue #531).
You'll need to kick it up to 100 meg or so in order to have your really large files deltified.
If you don't, each revision will be full-text, and take up 100 meg or whatever.
3. rcsparse is gonna be a little slow on those 552 meg files. At least make sure you have mxTextTools installed, so it uses the fast tokenizer.
--Dan
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Tue Jun 18 17:27:41 2002