[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: sporadic, hanging connections to apache svn (centos 5.4)

From: Nico Kadel-Garcia <nkadel_at_gmail.com>
Date: Tue, 16 Aug 2011 11:36:29 -0400

On Mon, Aug 15, 2011 at 11:08 PM, Seth Daniel
<subversion.org_at_sethdaniel.org> wrote:
> Hi,
>
> I've done many subversion installs but never have I seen the following
> problem before.
>
> Vanilla Centos 5.4.  Subversion 1.6.17 (compiled and packaged by me).
> Nothing fancy with the compilation.  I use an 'amalgamated' sqlite
> 3.6.23 when configuring...otherwise no other options to configure.  Then
> make; make install.  The client being used is from the same 1.6.17
> package.  Apache with prefork MPM (centos 5.4 comes with apache 2.2.3).

Dude!!! I'm building up an SRPM for RPMforge, backported from the
Fedora Core 16 test release of subversion-1.6.17. Would you care to
share notes,, or perhaps even test mine as scratch monkey
(http://www.catb.org/jargon/html/S/scratch-monkey.html) ?

I also encourage you to jump to CentOS 5.6: while it took a while to
come out, there are numerous small improvements that pay off at tough
moments, like the profound stability updates to autofs.

> The problem: Occasional and (so far) non-deterministic 'hangs'[1] of
> subversion requests.  It doesn't seem to matter if it's a commit or a
> status or a log.  Eventually, at some point, a request to the server
> will hang.  All requests are https.

That's no good!!!

> If I perform an strace on the client and an strace on the Apache child
> servicing the client they are both 'poll'ing.  I don't see any errors or
> warnings in the various apache logs.
>
> During the time this one request 'hangs' other requests are serviced
> just fine by the other apache children.
>
> I have read that apr sometimes needs to be compiled to use /dev/urandom.
> I am using the apr libs that come with Centos 5.4.  This is apparently
> apr 1.2.7.  Can an apr that uses /dev/random cause the problem I am
> describing?  I can find no evidence that the apr that comes with Centos
> 5.4 is compiled to use /dev/urandom, but I'm likely missing something
> obvious (I installed the apr-devel package and ran through the options
> for apr-1-config, such as --cppflags, and none mentioned a switch for
> compiling with /dev/urandom).

I really encourage you do update to CentOS 5.6 to help with underlying
architectural problems that may have been addressed two years ago by
the upstream vendor. I also assume that you're using the HTTPS based
access: can you test with svn:// or ssh+svn://

> If someone has any other ideas for figuring out what is going on I'm all
> ears and can do (most) any task.  One thing I have not been able to do
> is get it to consistently hang.  I have to run a request in a loop and
> eventually a request will hang.  Sometimes it takes 10 minutes,
> sometimes 1-2 hours...but eventually one of the requests will hang.
>
> Thanks folks.
>
>
> [1] it does not hang forever.  it simply takes a very long time to
>    respond.  I've seen as long as 45 minutes for a very simple log request
>    that just seconds earlier took about 1-2 seconds to return.

That...... is not good. I wonder if you do have an "I ran out of
randomness" issue? Can you roll back to the subversion-1.6.11 from
CentOs 5.6, or the subversion-1.6.15 from RPMforge (that I helped
bundle) on another server, and see if you can replicate the behavior?
Is this a wildly *busy* server?
Received on 2011-08-16 17:37:01 CEST

This is an archived mail posted to the Subversion Users mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.