[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: FSFS packing functionally complete

From: Branko Čibej <brane_at_xbc.nu>
Date: Sat, 29 Nov 2008 02:43:50 +0100

Mark Mielke wrote:
> If you are talking about other "more advanced" file systems such as
> reiserfs, they often offer tail packing, minimizing any gain from the
> block at the end of the file taking up extra space.
>
> FSFS packing of completed shards does a good job of dealing with this
> problem. Perhaps it isn't the common one people deal with every day
> (because their disks are usually much larger than the data they are
> putting on it and at least some commits are large), but inode
> exhausting is a real people that some people have experienced before.

Certainly you'll run into i-node limits in ext-derived filesystems, and
many "classic" -- what a word -- Unix filesystems. What I'm getting at
is that you don't need FSFS packing, nor sharding, to avoid those
problems these days; XFS, NTFS etc. that don't suffer from i-node limits
and use B-trees for directories don't generally suffer from the problems
those solutions are meant to fix.

Still, even on these filesystems you'll significantly reduce allocation
overhead by packing the files. And it could be that sharding improves
things for applications that aren't build with indexed directories in mind.

The best argument I know for sharding now is that you can easily scale
your storage by splitting your repository across volumes on the shard
level. Works like a charm on distributed filesystems, too.

-- Brane

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe_at_subversion.tigris.org
For additional commands, e-mail: dev-help_at_subversion.tigris.org
Received on 2008-11-29 02:44:20 CET

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.