On Wed, Jul 21, 2004 at 12:05:47AM +0200, Sven Mueller wrote:
> Anyway, what I really meant was: That person (you?) overlooked the
> pretty well known problems with huge directories. Making mistakes is
> human. However it is pretty difficult to change FSFS layout now without
> breaking existing installations.
Actually, here's a strawman plan:
At open time: determine whether we have a new format or old format
repository (based on, perhaps, the content of "format", or the existence
of new-format-only directories). If old format, convert to new format.
The conversion be essentially a few mkdirs and bunch of rename operations.
This could even be written as a shell script.
Afterwards:
Instead of putting revision N in revs/N, put it in revs(N/1000)/N.
This gives us 1000 files per revs dir, and doesn't use more than 1000
revs dirs until after 1000000 revs. The same thing goes for revprops.
If you've got a multi-million-rev repo, you've got bigger problems than
ls. :-)
We could design a fancier layout that maintains this upper bound up to
arbitrary revision counts, but I think that may be too far down the
complexity/return curve. It is entirely possible that by the time there
are repositories with several million revs, they will be on better
behaved filesystems.
I am opposed to hashing strategies that create a big forest of directories
(a.k.a. wasted inodes) and make it difficult for humans to quickly find
particular files. Also, deep hierarchies introduce additional filesystem
load in terms of directory inode traversal, etc.
--ben
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Wed Jul 21 00:27:32 2004