[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Possible memory leak in mod_dav_svn

From: Alexander Björnhagen <Alexander.Bjornhagen_at_nasdaqomx.com>
Date: Fri, 30 May 2008 09:38:14 +0200

It seems that mod_dav_svn has issues handling large operations with authorization enabled.
 
Our setup is a RHEL5 server with stock rpms :
httpd.x86_64->2.2.3-6.el5
mod_ssl.x86_64 -> 2.2.3-6.el5
mod_dav_svn.x86_64 -> 1.4.2-2.el5
subversion.x86_64 -> 1.4.2-2.el5
 
The relevant apache config is this:
 
<Location /repos>
   LimitRequestBody 0
   LimitXMLRequestBody 0
   DAV svn
   SVNParentPath /usr/local/svn/repos
   SVNListParentPath on
   SVNIndexXSLT /svnindex.xsl
   AuthType Basic
   AuthBasicProvider ldap
   AuthzLDAPAuthoritative off
   AuthzSVNAccessFile /etc/httpd/conf.d/subversion-authz
   AuthLDAPURL "ldap://edited:3268/DC=corp,DC=ad,DC=local?sAMAccountName?sub?(objectClass=*)" NONE #Microsoft ldap
   AuthLDAPBindDN "edited"
   AuthLDAPBindPassword edited
   require valid-user
</Location>
 
The authz file could be as simple as
[repo:/]
*=rw
 
The problem is that large checkouts *sometimes* fails with logs like this for https:
[Thu May 22 15:55:30 2008] [error] [client 10.175.105.17] Provider encountered an error while streaming a REPORT response. [500, #0]
[Thu May 22 15:55:30 2008] [error] [client 10.175.105.17] A failure occurred while driving the update report editor [500, #104]
[Thu May 22 15:55:30 2008] [error] [client 10.175.105.17] Error writing base64 data: Connection reset by peer [500, #104]
 
And entries like this for http:
[Thu May 22 15:59:35 2008] [error] [client 10.112.152.135] Provider encountered an error while streaming a REPORT response. [500, #0]
[Thu May 22 15:59:35 2008] [error] [client 10.112.152.135] A failure occurred while driving the update report editor [500, #190004]
 
On the client the error is typically:
 
svn: REPORT request failed on '/svn/repo/!svn/vcc/default'
svn: REPORT of '/svn/repo/!svn/vcc/default': Could not read chunk delimiter: Secure connection truncated (https://svn.myco.com <https://svn.myco.com/> )
 
We have tried to nail down why it only fails sometimes, but this is consistent accross networks and different clients etc. Local checkouts via file:/// always works.
 
After much googling the only thing that helps is setting:
 
SVNPathAuthz off
 
which is not good enough security wise for our purposes. I found out about this by reading this:
 
http://www.sharp-tools.net/archives/000928.html
 
which is the only reference to this exact issue that I can find. I tried to leave a post on that page to ask exactly how they found out that there was a memory leak but the comment system is broker (server 500 page :(
 
Another possible workaround would be to set up a cron job like this (at least it *seems* to work for us although I have not tested it long enough and definetly not put it into production... )
 
* * * * * /etc/init.d/httpd configtest 2>/dev/null && /etc/init.d/httpd graceful #guarantee that every http requset gets a newly started fresh thread
 
Seems I'm not the only one asking:
 
http://groups.google.se/group/linux.debian.bugs.dist/browse_thread/thread/d14c54c3b8a56255
http://svn.haxx.se/dev/archive-2008-02/0628.shtml
 
Anyway it makes sense that http worker threads leak memory and then at some point just die and drop their connections.
 
How can I verify the memory leaking and supply more concrete info ? ( I'd rather not sit and stare at top and hope to catch defunct processes since the errors occurs say five times per day )
 
Regards,
 
Alex B
 
Legal Disclaimer:
***********************************************************
This e-mail and the information it contains may be privileged and/or confidential.
It is for the intended addressee(s) only.
The unauthorized use, disclosure or copying of this e-mail, or any information it contains, is prohibited.
If you are not an intended recipient, please contact the sender and delete the material from your computer.
***********************************************************
Received on 2008-05-30 09:38:32 CEST

This is an archived mail posted to the Subversion Dev mailing list.