[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Sharded FSFS repositories - summary

From: Ph. Marek <philipp.marek_at_bmlv.gv.at>
Date: 2007-03-13 13:47:06 CET

On Tuesday 13 March 2007 13:00, Malcolm Rowe wrote:
> Here's a summary of the discussions about sharding FSFS repositories,
> and what I'd like to do for 1.5.0.
>
> Generally, everyone seems to like the idea. However, no-one wanted to
> be forced to pick a shard size, and it didn't look like anyone actually
> wanted the size to be configurable either ("the fewer decisions that
> have to be made at repository creation time the better").
...
> So, here's my plan so far. Any comments?
...
> - We'll create shards of 4000 entries each. That's large enough that
> someone would have to hit 16M revisions before a larger value would be
> an improvement, but small enough that it has reasonable performance
> (and support) on all filesystems that I'm aware of. It's also a
> power-of-ten, so easier for humans to understand.
4000 is no (integer) power of ten, so would not really be better.
Quick, in which directory is revision 421712? (see KDEs repository)

If I understand you correctly, you want to have
    0/1
    0/2
    0/3
    ...
    0/3999
    1/4000
    1/4001
    ...
    2/8000
and so on. Right?

I'd prefer to have a *real* (integer :-) power of ten, eg. 1000. And TBH, 4000
is a bit too much (for me, at least) - 1000 would be high, but acceptable.
(I'd really prefer 100 and 3 or 4 levels - but I seem to be alone with that.)

So naturally I'd like to have an option to specify that on creation time.

If this number would go into an existing file (format, fs-type), it would not
require another read; and if we allowed not one, but two such numbers here,
the repository could be re-arranged on-line.
(The fs-layer had to be looking for both files until one was found - as was
already recommended).

> - The revision files will be named according to the scheme I posted
> earlier, and shards created on-demand.
...
+1

Have you seen my mail regarding the transaction-directories? Maybe the naming
there could be done with the same function.

Thank you!

Regards,

Phil

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Tue Mar 13 13:47:26 2007

This is an archived mail posted to the Subversion Dev mailing list.