[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: svnserve under Linux inetd hangs, burning CPU cycles, under too low TCP-sendbuffer.

From: Dr. Andreas Krüger <andreas.krueger_at_dv-ratio.com>
Date: Mon, 01 Mar 2010 15:58:00 +0100

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello, Philip,

> I tried to reproduce your problem without using OpenVZ.

thanks for trying! (Though I don't think you have succeeded.)

To follow your request, while the stuck download was running and
getting nowhere, I did two "ls -laR" on the client's working
directory. The crucial big-stupid-binary-file.bin.svn-base was at a
bit of 3 MB before the 20 seconds and at the same bit over 3 MB
afterwards, with not a byte of progress having been made in 20
seconds. Find the precise information at the end of this message.

The entire single 10 MB file checkout normally happens in about 7 seconds.

I'm not complaining about "slow". I'm complaining about "no progress
at all" and "endless loop".
 
Also, I tend to see no openvpn activity, so apparently no serious
amount of network traffic.

Just to rule out openvpn as a source of trouble, this time, I have
used an ssh tunnel. I set it up via

ssh -L 7777:vzclientip:7777 root_at_vzhost

and used it via

cd /tmp; svn co localhost:7777 bug

resulting in the buggy behavior. This was actually the setup I used
for those two ls -laR .

For the record: The SVN server is back to version 1.5.1 from Debian Lenny.

With the server still spinning, I can kill the svn client.

Mild "kill"s don't lead to any results. But, of course, "kill -9" does
the trick.

After that, I wait for another 20 seconds.

Even now, 20 seconds after the death of the partner (that is, almost
three times longer then the entire check out normally takes), the
svnserve is still at 99.something % CPU load. It continues to be there
until I kill the server.

So, imho, that's your "holy grail" to seek ;-) . That is, I speculate
if you can reproduce "99+% CPU load without client", you have indeed
reproduced my problem.

For the record: In a different experiment leading nowhere, I have
tried a few-line Perl script as a server, generating junk data. The
client, also a Perl script, actually double-checked it got the precise
junk the server had sent (as, of course, should be the case with
TCP/IP). Both were derived from what's in "man perlipc" under the
heading "Internet TCP Clients and Servers". Between those two, there
was no problem transferring 50 MB of data, even with ridiculously low
tcpsendbuf. I think 50k:350k is what I used at that time. So, as I
said, this line of investigation lead nowhere.

Regards,

Andreas

- --
Dr. Andreas Krüger, Berater, DV-RATIO NORDWEST GmbH
andreas.krueger_at_dv-ratio.com
GPG/PGP Fingerprint 8063 4A9B 362D 4220 A546 14C1 EA19 AADC FD44 5EB7

DV-RATIO NORDWEST GmbH
Tel: +49 (0)211 / 577 996-0
Fax: +49 (0)211 / 577 996-26
http://www.dv-ratio.com <http://www.dv-ratio.com>
Sitz der Gesellschaft Habsburgerstraße 12, 40547 Düsseldorf
Registergericht Düsseldorf HRB 34330
USt-IdNr.: DE811321837
Steuer-Nr.: 809/44031
Geschäftsführung: Günter Gerstmann
Prokura: Trudbert Vetter, Uwe Wolfram

DV-RATIO - "Kompetenz und Zuverlässigkeit seit 1980"

$ LANG=C sh -c 'ls -laR bug; sleep 20; echo again; ls -laR bug'
bug:
total 12
drwxr-xr-x 3 andreas andreas 4096 Mar 1 15:18 .
drwxrwxrwt 16 root root 4096 Mar 1 15:20 ..
drwxr-xr-x 6 andreas andreas 4096 Mar 1 15:18 .svn

bug/.svn:
total 28
drwxr-xr-x 6 andreas andreas 4096 Mar 1 15:18 .
drwxr-xr-x 3 andreas andreas 4096 Mar 1 15:18 ..
- -r--r--r-- 1 andreas andreas 121 Mar 1 15:18 entries
- -rw-r--r-- 1 andreas andreas 0 Mar 1 15:18 lock
drwxr-xr-x 2 andreas andreas 4096 Mar 1 15:18 prop-base
drwxr-xr-x 2 andreas andreas 4096 Mar 1 15:18 props
drwxr-xr-x 2 andreas andreas 4096 Mar 1 15:18 text-base
drwxr-xr-x 5 andreas andreas 4096 Mar 1 15:18 tmp

bug/.svn/prop-base:
total 8
drwxr-xr-x 2 andreas andreas 4096 Mar 1 15:18 .
drwxr-xr-x 6 andreas andreas 4096 Mar 1 15:18 ..

bug/.svn/props:
total 8
drwxr-xr-x 2 andreas andreas 4096 Mar 1 15:18 .
drwxr-xr-x 6 andreas andreas 4096 Mar 1 15:18 ..

bug/.svn/text-base:
total 8
drwxr-xr-x 2 andreas andreas 4096 Mar 1 15:18 .
drwxr-xr-x 6 andreas andreas 4096 Mar 1 15:18 ..

bug/.svn/tmp:
total 20
drwxr-xr-x 5 andreas andreas 4096 Mar 1 15:18 .
drwxr-xr-x 6 andreas andreas 4096 Mar 1 15:18 ..
drwxr-xr-x 2 andreas andreas 4096 Mar 1 15:18 prop-base
drwxr-xr-x 2 andreas andreas 4096 Mar 1 15:18 props
drwxr-xr-x 2 andreas andreas 4096 Mar 1 15:18 text-base

bug/.svn/tmp/prop-base:
total 8
drwxr-xr-x 2 andreas andreas 4096 Mar 1 15:18 .
drwxr-xr-x 5 andreas andreas 4096 Mar 1 15:18 ..

bug/.svn/tmp/props:
total 8
drwxr-xr-x 2 andreas andreas 4096 Mar 1 15:18 .
drwxr-xr-x 5 andreas andreas 4096 Mar 1 15:18 ..

bug/.svn/tmp/text-base:
total 3104
drwxr-xr-x 2 andreas andreas 4096 Mar 1 15:18 .
drwxr-xr-x 5 andreas andreas 4096 Mar 1 15:18 ..
- -rw-r--r-- 1 andreas andreas 3170304 Mar 1 15:18
big-stupid-binary-file.bin.svn-base
again
bug:
total 12
drwxr-xr-x 3 andreas andreas 4096 Mar 1 15:18 .
drwxrwxrwt 16 root root 4096 Mar 1 15:20 ..
drwxr-xr-x 6 andreas andreas 4096 Mar 1 15:18 .svn

bug/.svn:
total 28
drwxr-xr-x 6 andreas andreas 4096 Mar 1 15:18 .
drwxr-xr-x 3 andreas andreas 4096 Mar 1 15:18 ..
- -r--r--r-- 1 andreas andreas 121 Mar 1 15:18 entries
- -rw-r--r-- 1 andreas andreas 0 Mar 1 15:18 lock
drwxr-xr-x 2 andreas andreas 4096 Mar 1 15:18 prop-base
drwxr-xr-x 2 andreas andreas 4096 Mar 1 15:18 props
drwxr-xr-x 2 andreas andreas 4096 Mar 1 15:18 text-base
drwxr-xr-x 5 andreas andreas 4096 Mar 1 15:18 tmp

bug/.svn/prop-base:
total 8
drwxr-xr-x 2 andreas andreas 4096 Mar 1 15:18 .
drwxr-xr-x 6 andreas andreas 4096 Mar 1 15:18 ..

bug/.svn/props:
total 8
drwxr-xr-x 2 andreas andreas 4096 Mar 1 15:18 .
drwxr-xr-x 6 andreas andreas 4096 Mar 1 15:18 ..

bug/.svn/text-base:
total 8
drwxr-xr-x 2 andreas andreas 4096 Mar 1 15:18 .
drwxr-xr-x 6 andreas andreas 4096 Mar 1 15:18 ..

bug/.svn/tmp:
total 20
drwxr-xr-x 5 andreas andreas 4096 Mar 1 15:18 .
drwxr-xr-x 6 andreas andreas 4096 Mar 1 15:18 ..
drwxr-xr-x 2 andreas andreas 4096 Mar 1 15:18 prop-base
drwxr-xr-x 2 andreas andreas 4096 Mar 1 15:18 props
drwxr-xr-x 2 andreas andreas 4096 Mar 1 15:18 text-base

bug/.svn/tmp/prop-base:
total 8
drwxr-xr-x 2 andreas andreas 4096 Mar 1 15:18 .
drwxr-xr-x 5 andreas andreas 4096 Mar 1 15:18 ..

bug/.svn/tmp/props:
total 8
drwxr-xr-x 2 andreas andreas 4096 Mar 1 15:18 .
drwxr-xr-x 5 andreas andreas 4096 Mar 1 15:18 ..

bug/.svn/tmp/text-base:
total 3104
drwxr-xr-x 2 andreas andreas 4096 Mar 1 15:18 .
drwxr-xr-x 5 andreas andreas 4096 Mar 1 15:18 ..
- -rw-r--r-- 1 andreas andreas 3170304 Mar 1 15:18
big-stupid-binary-file.bin.svn-base

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkuL1fQACgkQ6hmq3P1EXrcNVQCgjEAnAwePaiUiFYeAjyfVIvRP
YHsAoKakS5jXp/OIuGDtMVsRqn+Dj51J
=Sj+q
-----END PGP SIGNATURE-----
Received on 2010-03-01 15:58:40 CET

This is an archived mail posted to the Subversion Dev mailing list.