C.A.T.Magic wrote:
> Branko Čibej wrote:
>
>> C.A.T.Magic wrote:
>>
>>>
>>> Hi subversion dev's,
>>>
>>>
>>> some time ago I wondered why most svn commands appear to be
>>> very slow, no matter how small the operation is
>>> or even if it does nothing at all.
>>> this is especially problematic if running test cases
>>> that require to push many revisions.
>>>
>>> Gred Hudson pointed me to 'svn_sleep_for_timestamps'.
>>>
>>>
>>> today i found some time to verify this and added some
>>> debugoutput into the svn_sleep_for_timestamps function.
>>> this is indeed the reason for the 'slowness'.
>>>
>>> svn_sleep_for_timestamps ALWAYS waits 1.10 seconds.
>>
>>
>>
>> It does? Then that's a bug.
>
>
> ok it waits until the next 'second change' to be more precise,
> but if you chain many svn commands from a script, its
> effectively always an extra second.
Nope, it's half a second on average. :-)
>> Perhaps that's the problem we're seeing with FAT, yes. (It's not data
>> loss, though -- be careful about the terms you use. Data loss is a
>> _very_ serious issue, this one isn't.)
>
> I have no (FAT) filesystem available with only a 2 second resolution,
> but i could think of a situation like this
> ! imaginary example !:
>
> svn update
> #=> creates file1 with current time 00:00:00.10
> #=> FAT rounds this to 00:00:02
> #=> svn stores that time as 'last modified' in its entries file.
> #=> user modifies a character in the file no size change)
> # at 00:00:01.30 -- FAT rounds this to 00:00:02 again.
> svn commit
> #=> does nothing - file time hasn't changed, size hasn't changed
> #=> user thinks everything is committed and deletes his WC,
> or a release version is built without the required fix etc.
This is a very contrived example. First, it's very unlikely that a user
would do something like that in under 2 seconds. Second, if you mean
"script" instead of "user", well, a script that doesn't check the
command output is broken. The commit could faile because of a conflict,
for example.
> I'm trying to create test scripts that exactly 'replay' all user-actions
> that were performed to create our production repository, to test for
> svn's stability.
> this means i have about ~20.000 files that were added file-by-file
> over time, resulting in ~30.000 revisions.
> so, if i replay 30.000 svn actions this costs me - 30.000 seconds
> (>8 hours).
You created your production repository by committing one file at a time?
That's not the normal mode of usage for Subversion. Even if it is, and
it took on the order of 1 minute to _produce_ every one of those files,
the average wait time is 1% of the total.
> if i remove the sleep, test time goes down to less than half an hour.
> i know its opensource and i can change it at will, but i guess others
> may have similar situations/scripts with many calls to the svn
> executable and where this could significantly speed up things.
We're interested in _real_ usage situations where the sleep time
measurably affects the performance. From what you describe, all of your
tests are contrived and don't reflect actual usage patterns. You'd have
to take into account at least a) the fact that it takes some time to
edit a file and type "svn co", and b) the fact that usually you have
several users working in parallel.
>>> svn is so lightning fast if I remove the call to
>>> apr_sleep (then - now);
>>> or replace it with a smaller time period,
>>> I think you should really have a look into this.
>>
>> What is "lightning fast", and in which situations? If you update a
>> 40M working copy over the network, that additional second gets lost
>> in the noise. I suspect you're testing with very small, local-repos
>> data sets, which really aren't representative.
>
> my script mostly uses a sequence of
> svn add file1
> svn commit file1 *waits 1 sec*
> svn modify file2
> svn commit file2 *waits 1 sec*
> just like a normal user would perform it
No, this is _not_ how a normal user would perform it. A normal user
would do this:
<><> svn add file1 (takes about 2 seconds to type)
svn commit file1 (takes about 2 seconds, and another .5 second wait)
<> svn modify file2 (takes from 30 seconds to several hours)
svn commit file2 (takes about 2 seconds, and another .5 second wait)
So the actual time "lost" to waiting would be about 1 second out of,
say, 40. Do you need a 1.5% speedup here?
When optimizing performance, you must first make sure you measure the
right thing, and first optimize the things that take up a significant
amount of time. In the example above, you'd do better to send your users
to a typing course.
>> On your machine, with your particular test case. Try the stress.pl
>> test instead.
>
> If I find the time I'll ofcourse try that.
> does stress.pl work for Win32 ? i guess so. *need to get perl...*
Yes, it does.
-- Brane
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Fri Jun 25 15:52:32 2004