[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Poor performance with big tree (5K files)

From: Eric Kidd <eric.kidd_at_pobox.com>
Date: 2002-11-16 18:05:07 CET

Hello! I decided to download and mistreat your nifty VC system. I got
Subversion to misbehave in a variety of interesting ways, but I don't
know if the problems are specific to my system.

Cheers,
Eric

What I did:

  I installed subversion 0.14.5 on a Debian testing/unstable system,
  using 'apt-get -t unstable install subversion-client
  subversion-server'. I configured an Apache 2 server using prefork
  mode (the traditional threading model), and checked in a copy of the
  wxWindows 2.3.3 source distribution (which contains 5,000+ files).
  I executed 'svn cp' a few times to lay out my repository, and typed
  'svn co' on my newly-created trunk.

  My laptop has 256MB of RAM and an 800MHz Pentium III processor. The
  single '/' filesystem uses ext3 in ordered data mode.

What happened:

  Subversion took approximately 25 minutes to check out the tree, and
  rattled my disks like you wouldn't believe. A quick look at 'top'
  'M' revealed that 'svn' was consuming 50MB of RAM, and 7 Apache
  processes were consuming anywhere from 10 to 70MB of RAM:

  607 www-data 9 0 76880 66M 2648 S 0.0 26.7 0:44 apache2
  658 emk 17 0 56420 22M 2148 S 5.3 9.1 2:39 svn
  391 www-data 9 0 20716 14M 1932 S 0.0 5.8 0:30 apache2
  388 www-data 9 0 12676 11M 2032 S 0.0 4.7 0:30 apache2
  661 www-data 9 0 28132 10M 6064 S 0.0 4.1 0:31 apache2
  390 www-data 12 0 10756 10M 2368 R 2.3 4.0 0:23 apache2
  387 www-data 9 0 10472 9980 2040 S 0.0 3.9 0:26 apache2
  389 www-data 9 0 10116 9624 2032 S 0.0 3.7 0:23 apache2

  This is what a fresh Apache2 process looks like on my system:

  763 www-data 11 0 2372 2372 2284 S 0.0 0.9 0:00 apache2

  The Apache2 processes did not shrink after the checkout was
  complete. This is probably because Debian sets
  'MaxRequestsPerChild' to 0, keeping alive server processes for an
  indefinite amount of time.

  Interestingly enough, the Apache processes don't appear to grow much
  further if I do a second checkout. But I haven't investigated this
  all that thoroughly.

What should have happened:

  The checkout should have completed in 5 minutes or so, using a
  reasonable amount of RAM for both 'svn' and 'apache2'. When 'svn'
  was finished, the Apache processes should ideally have shrunk back
  to a manageable size.

My theories:

  Subversion's checkout speed is *much* slower than CVS's, but most of
  the difference seems to be on the wall clock. Subversion checkouts
  also generate a massive amount of disk I/O, even when the system has
  yet to touch the swap partition. Taken together, these facts
  suggest that somebody (perhaps APR) is calling sync(2) way too
  often.

  The massive bloat of Apache2 might be spurious--perhaps the database
  is mmaping big files--but svn is definitely doing something wrong.
  Unfortunately, both valgrind and mprof appear broken on my system,
  so I can't check for memory leaks.

Comparison to CVS:

  'svn update' and 'svn status' are faster than the corresponding CVS
  operations on this tree, but they still take anywhere from 3 to 30
  seconds, depending on how warm my caches are. 'svn import' and 'svn
  checkout' are slower than the corresponding CVS commands by nearly
  an order of magnitude.

  The inability of svn to recover after an aborted update makes this
  kind of testing rather awkward.

  Subversion shows tremendous promise, but it's still pretty awkward
  for big trees.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Mon Nov 18 16:41:24 2002

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.