[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: faster client pre-1.0: decrease number of files&folders in .svn

From: solo turn <soloturn99_at_yahoo.com>
Date: 2003-02-06 19:24:56 CET

--- Philip Martin <philip@codematters.co.uk> wrote:
> solo turn <soloturn99@yahoo.com> writes:
> > > "Slow down"? To what are you comparing it?
> >
> > how it scales with the number of files/folders managed. this is
> not
> > linear. its minimum 2 times slower (ie 10 times more files
> managed,
> > 20 times more time for "svn st").
> [...]
> > 1. scalability of svn up and svn st:
> > our experience states O(>2), while we believe it
> > should be O(1). you may be partially right, because
> > it is not O(10) like the number of files suggest.
> > but i don't know how many .svn entries are
> > really touched for one file in the repository.
> > --> shows it is non-linear
> Looks like something other than Subversion is causing your
> problems, I
> see the expected linear behaviour

i can confirm these numbers. it seems the number of folders is
important. and our real world test numbers had a folder increase, so
it looked non-linear.

i ran the following cases:

1. files per folder: 100
  tree depth 2, #nodes: 1010, #files: 1000, #folders: 10
  $ tools/dev/stress.pl -c -F100 -D9 -N2 -n0

        $ time svn up -r1 wc > /dev/null
        real 1m46.865s
        user 0m19.870s
        sys 0m4.640s
        $ time svn st wc
        real 0m1.013s
        user 0m0.690s
        sys 0m0.290s
        $ time svn up -r4 wc > /dev/null
        real 7m18.664s
        user 1m22.830s
        sys 0m20.100s
        $ time svn st wc
        real 0m4.037s
        user 0m2.750s
        sys 0m1.120s

2. files per folder: 10
  tree depth 2, #nodes: 1010, #files: 1000, #folders: 100
  $ tools/dev/stress.pl -c -F10 -D99 -N2 -n0

        $ time svn up -r1 wc > /dev/null
        real 2m42.612s
        user 0m15.340s
        sys 0m5.270s
        $ time svn st wc
        real 0m3.467s
        user 0m1.010s
        sys 0m0.360s
        $ time svn up -r4 wc > /dev/null
        real 11m31.297s
        user 0m59.890s
        sys 0m23.660s

        $ time svn st wc
        real 0m5.875s
        user 0m3.610s
        sys 0m1.570s

3. files per folder: 1
  tree depth 4, #nodes: 1096, #files: 512, #folders: 584
  $ tools/dev/stress.pl -c -F1 -D8 -N4 -n0

        $ time svn up -r1 wc > /dev/null
        real 10m39.841s
        user 0m13.900s
        sys 0m10.380s

        $ time svn st wc
        real 0m3.859s
        user 0m2.090s
        sys 0m0.950s

        $ time svn up -r4 wc > /dev/null
        real 45m9.121s
        user 0m55.690s
        sys 0m39.410s

        $ time svn st wc
        real 0m17.802s
        user 0m10.040s
        sys 0m3.940s

these cases were "local", so no proxy, no ssl, no auth files touched
for an update. in this case it seems to hehave like:
- 22 min for 1000 folders (which is excessive imo)
- 1.5 min for 1000 files
i.e. one folder is 15 times as costly for "svn up" than a file.

(this was a server with other database on it so don't take the
numbers as exact measures ... just the 45min-4000folder case was run
when not doing much else).

Do you Yahoo!?
Yahoo! Mail Plus - Powerful. Affordable. Sign up now.

To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Thu Feb 6 19:25:52 2003

This is an archived mail posted to the Subversion Dev mailing list.