[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Problem Loading Huge Repository

From: Thomas Harold <thomas-lists_at_nybeta.com>
Date: Fri, 17 Jun 2011 11:50:45 -0400

On 6/17/2011 10:54 AM, Daniel Shahaf wrote:
> Thomas Harold wrote on Fri, Jun 17, 2011 at 10:31:43 -0400:
>> And if you have a choice of file systems for the repository to be stored
>> on, make sure that it's something which can deal with a few hundred
>> thousand tiny files. On Linux, I'd suggest going with ext4 over ext3.
>> While db/revs in a FSFS repository can have its revisions packed to
>> reduce the file count, the db/revprops folder still consists of 1 tiny
>> file for every revision in the project in a FSFS repository.
>
> revprops/ is sharded.
>
> And in 1.7 (including the recent 1.7.0-alpha1) it is packed, too.
>

Good. Another of the many reasons that we're looking forward to 1.7.

Even with the sharding, those little revprop files are causing us issues
during backups (hotcopy -> rdiff-backup). Being able to pack those
revprop files is going to make a big difference as the backup process
will only have to track 2000-2200 files instead of 30,000 to 50,000.

(We have a few long-lived repositories with up to 25k revisions. And I
just finished splitting a 22GB repository with 15-16k revs into a bunch
of smaller repositories. Now the nightly backup can look at doing a
hotcopy on only the repositories with changes in the last 5 days.)
Received on 2011-06-17 17:51:23 CEST

This is an archived mail posted to the Subversion Users mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.