Re: notification output bottleneck

On Thu, Aug 30, 2012 at 12:19 PM, Stefan Fuhrmann
<stefan.fuhrmann_at_wandisco.com> wrote:
> On 8/30/12, Johan Corveleyn <jcorvel_at_gmail.com> wrote:
>> On Wed, Aug 29, 2012 at 4:22 AM, Justin Erenkrantz
>> <justin_at_erenkrantz.com> wrote:
>>> On Tue, Aug 28, 2012 at 7:22 PM, Johan Corveleyn <jcorvel_at_gmail.com>
>>> wrote:
>>>> Yep, redirecting to a file eliminates the bottleneck (almost the same
>>>> as redirecting to NUL) (I ran it a couple of times to make sure the
>>>> server cache was hot):
>>>
>>> FWIW, I've historically seen similar behavior on Unix platforms as
>>> well - especially on machines with SSDs and a fast local network as
>>> the stdout I/O to emit the notifications is the slowest part of the
>>> system by far. -- justin
>>
>> Hmz, so contrary to what I thought it seems it's not only a problem on
>> Windows. Is is as severe on *nix as on Windows? My export (w/ fast
>> server over a LAN) was twice as fast when redirecting notifications to
>> a file. Can somebody get some numbers on some unixy platform?
>>
>> But more to the point: anybody have a solution in mind? If it's not
>> Windows-only then some Windows defines wont help of course. Buffering
>> the output may be the only way to eliminate this bottleneck? What are
>> the pros and cons, and how hard would that be? Any other ideas?
>
> Yep. The problem only manifests if the user wants
> to see the output in "real time" - not for later analysis.
> In the latter case, the user would have redirected the
> output somewhere else.
>
> So, how about a '--summary' option for export / co / up
> reports that aggregates consecutive lines if they refer to
> the same folder, e.g. /some/folder/path 12 added 13 updated.
> In case of an error, the detailed list for that folder would
> be displayed.
>
> Please note that there may be multiple summary outputs
> per folder as the incoming data is not necessarily sorted
> by path, IIRC.

Hm, I'm not really in favor of that (too complicated).

Also, the problem is that printing out the notifications is the
default behavior, so most of the time it's not even a conscious
decision to want to see all output in real time. Most users will
simply type 'svn checkout $URL/huge-project', and then wait for 50
minutes while tens of thousands of notification lines fly by. Little
do they know that their checkout would only take 25 minutes if they
had only used the -q option. It's an quick win in performance, but 99%
of users don't know about it.

-- 
Johan