[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Subversion slows down with large number of files?

From: Julian Foad <julianfoad_at_btopenworld.com>
Date: 2004-10-01 18:44:04 CEST

Jaap Vermeulen wrote:
>
> I'm running 1.1.0 RC 3 using the svnserve server on one machine, and use the
> svn: protocol on the client machine.
>
> Everything has been working fine until I hit a directory with approx. 10332
> files in it (small HTML files, combined total size of 18.2 Mb). This seems
> to bog down SVN (or Tortoise SVN 1.1.0 RC2) to an enormous extent. If I use
> the command line svn command, it will eat up close to 100% CPU time for 10
> minutes or so on a commit. If I use the RepoBrowser and try to expand the
> directory with those files, the TortoiseProc will eat up 100% for a long
> time. If I try to open or refresh an explorer window, it goes to 100% for a
> couple of minutes.
>
> Is this to be expected? Doesn't seem like it should cause such a dramatic
> slowdown.

It is a known problem - that Subversion is slow when using the BDB back-end and there are very many files in the same directory. There doesn't seem to be an entry for it in the issue tracker. Please could you file one?

In the thread "Perf issues with BDB and directories with a large number of items",
Branko Čibej wrote:
> Brian W. Fitzpatrick wrote:
>
>> Summary: If you have a directory in your repository with a large number
>> of items in it, the BDB backend gets slower as you add more items (due
>> parsing/unparsing skels I'll bet).
>>
>> FSFS does not exhibit this behavior.
>>
>> Details and Pretty Pictures here: http://www.red-bean.com/fitz/svn/
>>
> This is a textbook example of quadratic vs. logarithmic behaviour, and
> it's a safe bet that it's caused by our saving directory data in
> unsorted lists and rewriting the whole list at every directory change.

- Julian

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Fri Oct 1 18:45:25 2004

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.