Re: svn ls performance

On 05.01.2012 01:25, Philip Martin wrote:
> Konstantin Kolinko<knst.kolinko_at_gmail.com> writes:
>
>> 2012/1/5 Stefan Küng<tortoisesvn_at_gmail.com>:
>>> Hi,
>>>
>>> Due to a report on the TSVN mailing list I found that the CL client has the
>>> same problem:
>>> 'svn list' takes forever in some situations.
>>> I don't know what the problem exactly is, but it's easily reproducable:
>>>
>>> svn ls http://plugins.svn.wordpress.org/ -v --depth=immediates
>>> prints one entry, then never returns (ok, maybe not never. But waiting 10
>>> minutes is not enough).
>>>
>>> however, an
>>> svn ls http://plugins.svn.wordpress.org/ --depth=immediates
>>> (same command as above, but without the verbose flag) returns the entries
>>> almost immediately.
>>> I don't think that simply fetching the verbose info could take so much
>>> longer?
>>>
>>> This is with svn 1.7.2.
>>> The same works fine with svn 1.6.6 - haven't tested with later 1.6 svn
>>> clients since I don't have that version ready here on my machine.
>>>
>>> using serf instead of neon doesn't help.
>>>
>>
>> http://plugins.svn.wordpress.org/
>> The page has a lot of subdirectories (nearly 26000)
>> The server is 1.6.12.
>>
>> Using client 1.6.17 (built by CollabNet, 32-bit, on Windows) it
>> printed the first line in 10 seconds, and never printed the rest. I
>> waited for ~1 minute. (Ctrl+C did not help, I had to unplug the
>> network cable)
>>
>> Using client 1.7.2 (from TortoiseSVN, 32-bit, on Windows) the result
>> is the same:
>> the first line in ~10 seconds, the rest of data - ~never.
>>
>> 1.6.17: Removing "--depth" option does not change anything.
>> 1.6.17: Removing "-v" the result appears in ~15 seconds and then takes
>> ~10 seconds to print the listing to the console.
>
> I created a repository with 10,000 subdirs:
>
> #!/bin/bash
> for i in `seq 0 999`;do
> svn mkdir -mm file://`pwd`/repo/A${i}{0,1,2,3,4,5,6,7,8,9}
> done
>
> I measured the following times:
>
> client server ls -v ls
> 1.6.6 1.6.12 43s 0.3s
> 1.6.12 1.6.12 43s 0.3s
> 1.7.3 1.6.12 43s 0.3s
> 1.6.6 1.7.3 1.8s 0.5s
> 1.6.12 1.7.3 1.8s 0.4s
> 1.7.3 1.7.3 1.8s 0.4s
>
> I don't see any significant difference between clients (I'm using neon)
> only between the servers. 1.7 is a major improvement in the verbose
> case, probably due to the better FSFS in-memory caching. There is
> perhaps a slight regression in the non-verbose case.
>
> As a side issue having 26,000 branches in the same directory is really
> bad for repository size due to the absence of directory deltification.
> My repository has 10,000 subdirs in 1,000 revisions and nothing else and
> yet it takes 175MB of disk. The last commit, which adds 10 empty
> subdirs, produces a rev file that is 347KB. Each commit to the
> wordpress repository probably adds about 1MB to the repository just
> rewriting that 26,000 branch directory.
>

Hmm - strange. I've had significant different timings for 1.6.6 and 1.7.2.
But I've tried 1.6.6 right after 1.7.2, so maybe there was some caching
involved?

I'll have to do some more testing.

But could the performance be somewhat improved? After all, without the
verbose flag the result is available much, much faster.

Stefan

-- 
        ___
   oo  // \\      "De Chelonian Mobile"
  (_,\/ \_/ \     TortoiseSVN
    \ \_/_\_/>    The coolest Interface to (Sub)Version Control
    /_/   \_\     http://tortoisesvn.net