[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Sharded FSFS repositories - summary

From: John Szakmeister <john_at_szakmeister.net>
Date: 2007-03-17 10:42:30 CET

Justin Erenkrantz wrote:
> On 3/15/07, John Peacock <jpeacock@rowman.com> wrote:
>> Of course, knowing this, as a rule I never open large directories in
>> Explorer, but use a Command Prompt instead. It's still painfully slow
>> to get a directory here, because DIR insists on sorting the files
>> (rather than returning them in filesystem order).
>
> And, doing an ls in a directory 500k+ files in it even on Unix is no
> fun either. I think we're sort of straying from the point here - for
> those high-volume repositories (like Apache, etc.), sharding is a way
> to reduce inode exhaustion in directories - not eliminate the issues.
> 5k (just to keep it power of 10) seems like a good cut-off. 1k is far
> too small as apache.org is going to zoom by 1 million revs soon
> enough.

I think there is more applicability than that. In particular, I think
sharding can help folks who like to keep their repository on usb keys,
since most of them are formatted with FAT (which is very limited, in
terms of entries per directory). But you're right, the primary
motivation is reducing inode exhaustion. Mea culpa.

If we're going to make this configurable, with a reasonable default, can
we default with something that's more reasonable for typical and mobile
users? And for those who are clearly out of the norm, such as Apache,
they can specify something more suitable to their environment?

> So, in other words, I couldn't care less about what the folders look
> like on Win32 - to focus on that exclusively is to be beside the point
> - *most* serious large-scale repositories probably aren't going to be
> on Win32. They can, but then those admins aren't likely to be foolish
> enough to browse the directories with Explorer on a regular basis - I
> claim that we should assert that whomever is admining that large of a
> repository probably has a modicum of clue to understand what's going
> on here.
>
> My $.02. -- justin

I really don't want to draw this out any further, so I won't post
anymore on it. If aren't worried about repositories on USB keys (and we
haven't to date), then I think 5k is very reasonable.

-John

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Sat Mar 17 10:43:25 2007

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.