[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: FSFS packing functionally complete

From: David Glasser <glasser_at_davidglasser.net>
Date: Sat, 29 Nov 2008 14:11:14 -0500

On Sat, Nov 29, 2008 at 2:05 PM, Hyrum K. Wright
<hyrum_wright_at_mail.utexas.edu> wrote:
> Hyrum K. Wright wrote:
>> Hi all.
>>
>> As of r34446, the implementation of packing on fsfs is functionally complete on
>> the fsfs-pack branch. For those that don't know, packing consists of mushing
>> all the individual rev files in a completed shard into one file, thus
>> eliminating the inode penalty for that entire shard. Packing a trunk-generated
>> copy of the ASF repository saved about 1 GB on a 24 GB repo. There may be
>> additional performance benefits in dealing with a much smaller set of rev files
>> (OS caching, etc.), but I haven't yet investigated that.
>>
>> This comes at a cost: the offsets of revisions in the pack file are stored
>> separately, and thus require an additional open/seek to get that information.
>> Also, determining whether a revision is stored in a pack file or not also
>> requires additional I/O. I think that most of this can be eliminated with
>> caching and heuristics, but those haven't yet been implemented.
>
> As of r34473, packed revision range information and offset caching has been
> implemented. I believe this makes fsfs-packing not only more space-conscious
> than and unpacked repository, but it also performs as well (once the caches are
> properly populated).
>
>> I'm not currently planning on including this functionality in 1.6, as it's kinda
>> biggish feature, the optimizations aren't yet in place, and I feel like merging
>> right before we branch 1.6.x could be a bit destabilizing. However, I could
>> easily be talked into it. :)
>
> Given that this is an optional feature, the positive feedback I've heard onlist
> has convinced me that I should merge this to trunk soon. Since it's been a
> holiday weekend (in the US), I won't merge until Monday some time, to allow
> people to provide any objections.

How about updating 'structure' before merging?

--dave

-- 
David Glasser | glasser@davidglasser.net | http://www.davidglasser.net/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe_at_subversion.tigris.org
For additional commands, e-mail: dev-help_at_subversion.tigris.org
Received on 2008-11-29 20:11:29 CET

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.