Re: notification output bottleneck

On Mon, Aug 27, 2012 at 10:45 AM, Johan Corveleyn <jcorvel_at_gmail.com> wrote:
> On Mon, Aug 27, 2012 at 2:09 AM, Bert Huijben <bert_at_qqmail.nl> wrote:
>>> -----Original Message-----
>>> From: Johan Corveleyn [mailto:jcorvel_at_gmail.com]
>>> Sent: maandag 27 augustus 2012 00:39
>>> To: Subversion Development
>>> Subject: notification output bottleneck
>>>
>>> I've never noticed this before (slow server, but now I'm testing a new,
>> faster
>>> server), but it seems that the writing of notification output on stdout is
>> a
>>> bottleneck for checkout, update or export on Windows (cmd.exe). With a
>>> fast server, hot caches, and everything on a Gb LAN, checkout of a large
>>> working copy is twice as fast (from 79 minutes down to 35 minutes) when I
>>> redirect output to NUL. I tested with both 1.7.5 (SlikSVN) and trunk, the
>>> results were about the same.
>>>
>>> Is there anything that could be done in svn to remove that bottleneck
>>> (buffering, ...)?
>>>
>>> (there might have been some discussion about this before, but I can't find
>> it)
>>
>> I found this problem too while profiling.
>>
>> The console apis on Windows in the Microsoft libraries are slow because they
>> try to be safe for multithreaded applications. There are some defines that
>> can be used to speed this up (which will avoid the most costly thread
>> synchronisations), but we can't just apply these to our code as that would
>> make libsvn_subr no longer multithread safe.
>
> Interesting.
>
> Am I correct in saying that this is really only a problem for the
> commandline 'svn' client (and perhaps 'svnadmin' and 'svnlook' etc
> ...)? Other clients can do their own fast notification handling,
> right?
>
> And Isn't 'svn' always single-threaded?
>
> In that case: couldn't we make the necessary multithread-unsafe, but
> fast, output functions for 'svn' only?

Is the above idea possible / sane / desirable?

Bert, what are those defines to speed up console access at the expense
of thread safety? I'd like to give them a quick try just to see how
much it would impact.

ISTR from before we migrated to SVN that CVS did not have this
problem: console output would just fly by very quickly. At least
that's how I remember it :-), I haven't run CVS for some time now
(it's also possible that I am misremembering, or that it is because I
was running the client on WinXP back then).

-- 
Johan