On 6/12/06, Jon Smirl <jonsmirl@gmail.com> wrote:
> I've been doing some experiments with importing the Mozilla CVS
> repository in to svn and git. I am having terrible performance
> problems with this process. Many of the tasks take days to complete.
>
> After importing Mozilla in to svn using cvs2svn, a big problem is that
> my svn repository has 450,000 files in it. Two directories have
> 220,000 files each in them. ext3 collapses under that size of
> directory since it does sequential looks ups for file names.
>
> It there some way to pack the repository down to fewer files?
>
> svn could be changed to get around this problem. The files are
> sequentially numbered from 1 to 220,000. It would be easy to put the
> first 1000 in one directory and so on to spread the files over 220
> directories. git uses this technique.
For what it's worth, I know of repositories that use fsfs and that are
way beyond that number of revisions. svn.apache.org/repos/asf is at
413k at the moment, and is doing fine. As others have suggested,
there may be kernel or fs parameters that you can tweak to avoid the
linear search problem, or switching to a non-ext3 filesystem may be an
option.
If you really feel that the number of files is a problem, feel free to
come to the dev@ list with a proposal on how to fix it. There are
various hashing schemes that could be tried, it's just that nobody's
been sufficiently motivated to try it since it's usually easier to
just tweak the filesystem to make it fast enough.
-garrett
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Received on Mon Jun 12 19:43:19 2006