Re: svn ls performance

Konstantin Kolinko <knst.kolinko_at_gmail.com> writes:

> 2012/1/5 Stefan Küng <tortoisesvn_at_gmail.com>:
>> Hi,
>>
>> Due to a report on the TSVN mailing list I found that the CL client has the
>> same problem:
>> 'svn list' takes forever in some situations.
>> I don't know what the problem exactly is, but it's easily reproducable:
>>
>> svn ls http://plugins.svn.wordpress.org/ -v --depth=immediates
>> prints one entry, then never returns (ok, maybe not never. But waiting 10
>> minutes is not enough).
>>
>> however, an
>> svn ls http://plugins.svn.wordpress.org/ --depth=immediates
>> (same command as above, but without the verbose flag) returns the entries
>> almost immediately.
>> I don't think that simply fetching the verbose info could take so much
>> longer?
>>
>> This is with svn 1.7.2.
>> The same works fine with svn 1.6.6 - haven't tested with later 1.6 svn
>> clients since I don't have that version ready here on my machine.
>>
>> using serf instead of neon doesn't help.
>>
>
> http://plugins.svn.wordpress.org/
> The page has a lot of subdirectories (nearly 26000)
> The server is 1.6.12.
>
> Using client 1.6.17 (built by CollabNet, 32-bit, on Windows) it
> printed the first line in 10 seconds, and never printed the rest. I
> waited for ~1 minute. (Ctrl+C did not help, I had to unplug the
> network cable)
>
> Using client 1.7.2 (from TortoiseSVN, 32-bit, on Windows) the result
> is the same:
> the first line in ~10 seconds, the rest of data - ~never.
>
> 1.6.17: Removing "--depth" option does not change anything.
> 1.6.17: Removing "-v" the result appears in ~15 seconds and then takes
> ~10 seconds to print the listing to the console.

I created a repository with 10,000 subdirs:

#!/bin/bash
for i in `seq 0 999`;do
  svn mkdir -mm file://`pwd`/repo/A${i}{0,1,2,3,4,5,6,7,8,9}
done

I measured the following times:

client server ls -v ls
1.6.6 1.6.12 43s 0.3s
1.6.12 1.6.12 43s 0.3s
1.7.3 1.6.12 43s 0.3s
1.6.6 1.7.3 1.8s 0.5s
1.6.12 1.7.3 1.8s 0.4s
1.7.3 1.7.3 1.8s 0.4s

I don't see any significant difference between clients (I'm using neon)
only between the servers. 1.7 is a major improvement in the verbose
case, probably due to the better FSFS in-memory caching. There is
perhaps a slight regression in the non-verbose case.

As a side issue having 26,000 branches in the same directory is really
bad for repository size due to the absence of directory deltification.
My repository has 10,000 subdirs in 1,000 revisions and nothing else and
yet it takes 175MB of disk. The last commit, which adds 10 empty
subdirs, produces a rev file that is 347KB. Each commit to the
wordpress repository probably adds about 1MB to the repository just
rewriting that 26,000 branch directory.

-- 
uberSVN: Apache Subversion Made Easy
http://www.uberSVN.com