[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: svnserve under Linux inetd hangs, burning CPU cycles, under too low TCP-sendbuffer.

From: Philip Martin <philip.martin_at_wandisco.com>
Date: Mon, 01 Mar 2010 13:26:06 +0000

"Dr. Andreas Krüger" <andreas.krueger_at_dv-ratio.com> writes:

> I also see
> several poll timeouts
>
> poll([{fd=0, events=POLLIN}], 1, 0) = 0 (Timeout)
>
> and several successful brk calls, before it finally all starts over again.
>
> More or less over again, that is. There seems to be an _llseek into
> that revision file that seeks differently each time. The precise
> numbers do not grow, they fluctuate.

I tried to reproduce your problem without using OpenVZ. On my Linux
desktop I set:

  $ sudo sysctl -w net.ipv4.tcp_wmem='256 512 1024'

and then I used my laptop to checkout a working copy containing a 21MB
file. It's very slow but it does succeed. The client doesn't give
much feedback during a very slow checkout, but if you look in
.svn/tmp/text-base you can see the temporary text bases getting
bigger.

The svnserve process doesn't use much cpu, it's mostly waiting. An
strace -T shows:

brk(0x17c7000) = 0x17c7000 <0.000015>
brk(0x17b7000) = 0x17b7000 <0.000029>
brk(0x17a7000) = 0x17a7000 <0.000020>
brk(0x17a5000) = 0x17a5000 <0.000013>
poll([{fd=4, events=POLLIN}], 1, 0) = 0 (Timeout) <0.000010>
poll([{fd=4, events=POLLIN}], 1, 0) = 0 (Timeout) <0.000009>
poll([{fd=4, events=POLLIN}], 1, 0) = 0 (Timeout) <0.000009>
write(4, " ) ) ( textdelta-chunk ( 2:c1 9:\0"..., 4096) = 4096 <0.076367>
write(4, "S\371\266\370\321\364\3540+0\251\260m`\204\244\200\0210{>g:2\21I\343*\307foR\305"..., 98422) = 98422 <2.678045>
read(6, "?\v\3\350+3P\264\210\3%\344g\3115\4a\203\247\301\31\227B\307\257IC\260@\177\35\230q"..., 4096) = 4096 <0.000018>

followed by several more reads and then it repeats. As you can see
it's the second write call that takes a long time (2.7s), which isn't
really surprising since it's a big write with a small buffer. If I
bump up the tcp_wmem values that time goes down to a tiny fraction of
a second.

> I finally increase this vz slice's tcpsndbuf allowance to
> 614400:921600 and, voila, that solves the problem.

Is there a bug here? If you use small buffers the writes will be
slow. Is the svnserve process using significant cpu or is it just
waiting for the writes to complete? If you look in .svn/tmp/text-base
is the client making progress downloading the text bases?

-- 
Philip
Received on 2010-03-01 14:26:45 CET

This is an archived mail posted to the Subversion Dev mailing list.