Re: svn ls performance

On Thu, Jan 5, 2012 at 22:02, Stefan Küng <tortoisesvn_at_gmail.com> wrote:
> On 05.01.2012 01:25, Philip Martin wrote:
>>
>> Konstantin Kolinko<knst.kolinko_at_gmail.com>  writes:
>>
>>> 2012/1/5 Stefan Küng<tortoisesvn_at_gmail.com>:
>>>>
>>>> Hi,
>>>>
>>>> Due to a report on the TSVN mailing list I found that the CL client has
>>>> the
>>>> same problem:
>>>> 'svn list' takes forever in some situations.
>>>> I don't know what the problem exactly is, but it's easily reproducable:
>>>>
>>>> svn ls http://plugins.svn.wordpress.org/ -v --depth=immediates
>>>> prints one entry, then never returns (ok, maybe not never. But waiting
>>>> 10
>>>> minutes is not enough).
>>>>
>>>> however, an
>>>> svn ls http://plugins.svn.wordpress.org/ --depth=immediates
>>>> (same command as above, but without the verbose flag) returns the
>>>> entries
>>>> almost immediately.
>>>> I don't think that simply fetching the verbose info could take so much
>>>> longer?
>>>>
>>>> This is with svn 1.7.2.
>>>> The same works fine with svn 1.6.6 - haven't tested with later 1.6 svn
>>>> clients since I don't have that version ready here on my machine.
>>>>
>>>> using serf instead of neon doesn't help.
>>>>
>>>
>>> http://plugins.svn.wordpress.org/
>>> The page has a lot of subdirectories (nearly 26000)
>>> The server is 1.6.12.
>>>
>>> Using client 1.6.17 (built by CollabNet, 32-bit, on Windows) it
>>> printed the first line in 10 seconds, and never printed the rest. I
>>> waited for ~1 minute. (Ctrl+C did not help, I had to unplug the
>>> network cable)
>>>
>>> Using client 1.7.2 (from TortoiseSVN, 32-bit, on Windows) the result
>>> is the same:
>>> the first line in ~10 seconds, the rest of data - ~never.
>>>
>>> 1.6.17: Removing "--depth" option does not change anything.
>>> 1.6.17: Removing "-v" the result appears in ~15 seconds and then takes
>>> ~10 seconds to print the listing to the console.
>>
>>
>> I created a repository with 10,000 subdirs:
>>
>> #!/bin/bash
>> for i in `seq 0 999`;do
>>   svn mkdir -mm file://`pwd`/repo/A${i}{0,1,2,3,4,5,6,7,8,9}
>> done
>>
>> I measured the following times:
>>
>> client   server     ls -v   ls
>> 1.6.6    1.6.12     43s     0.3s
>> 1.6.12   1.6.12     43s     0.3s
>> 1.7.3    1.6.12     43s     0.3s
>> 1.6.6    1.7.3      1.8s    0.5s
>> 1.6.12   1.7.3      1.8s    0.4s
>> 1.7.3    1.7.3      1.8s    0.4s
>>
>> I don't see any significant difference between clients (I'm using neon)
>> only between the servers.  1.7 is a major improvement in the verbose
>> case, probably due to the better FSFS in-memory caching. There is
>> perhaps a slight regression in the non-verbose case.
>>
>> As a side issue having 26,000 branches in the same directory is really
>> bad for repository size due to the absence of directory deltification.
>> My repository has 10,000 subdirs in 1,000 revisions and nothing else and
>> yet it takes 175MB of disk.  The last commit, which adds 10 empty
>> subdirs, produces a rev file that is 347KB.  Each commit to the
>> wordpress repository probably adds about 1MB to the repository just
>> rewriting that 26,000 branch directory.
>>
>
> Hmm - strange. I've had significant different timings for 1.6.6 and 1.7.2.
> But I've tried 1.6.6 right after 1.7.2, so maybe there was some caching
> involved?
>
> I'll have to do some more testing.
>
> But could the performance be somewhat improved? After all, without the
> verbose flag the result is available much, much faster.
>
>
Yes, performance of svn ls and svn ls -v was improved in svn 1.7.0.
See r1125391 and r1125326. Btw I'd like to '-v' be switched off by
default in TortoiseSVN Repo-Browser to make it faster.

-- 
Ivan Zhakov