It seems that mod_dav_svn has issues handling large operations with authorization enabled.
Our setup is a RHEL5 server with stock rpms :
httpd.x86_64->2.2.3-6.el5
mod_ssl.x86_64 -> 2.2.3-6.el5
mod_dav_svn.x86_64 -> 1.4.2-2.el5
subversion.x86_64 -> 1.4.2-2.el5
The relevant apache config is this:
<Location /repos>
LimitRequestBody 0
LimitXMLRequestBody 0
DAV svn
SVNParentPath /usr/local/svn/repos
SVNListParentPath on
SVNIndexXSLT /svnindex.xsl
AuthType Basic
AuthBasicProvider ldap
AuthzLDAPAuthoritative off
AuthzSVNAccessFile /etc/httpd/conf.d/subversion-authz
AuthLDAPURL "ldap://edited:3268/DC=corp,DC=ad,DC=local?sAMAccountName?sub?(objectClass=*)" NONE #Microsoft ldap
AuthLDAPBindDN "edited"
AuthLDAPBindPassword edited
require valid-user
</Location>
The authz file could be as simple as
[repo:/]
*=rw
The problem is that large checkouts *sometimes* fails with logs like this for https:
[Thu May 22 15:55:30 2008] [error] [client 10.175.105.17] Provider encountered an error while streaming a REPORT response. [500, #0]
[Thu May 22 15:55:30 2008] [error] [client 10.175.105.17] A failure occurred while driving the update report editor [500, #104]
[Thu May 22 15:55:30 2008] [error] [client 10.175.105.17] Error writing base64 data: Connection reset by peer [500, #104]
And entries like this for http:
[Thu May 22 15:59:35 2008] [error] [client 10.112.152.135] Provider encountered an error while streaming a REPORT response. [500, #0]
[Thu May 22 15:59:35 2008] [error] [client 10.112.152.135] A failure occurred while driving the update report editor [500, #190004]
On the client the error is typically:
svn: REPORT request failed on '/svn/repo/!svn/vcc/default'
svn: REPORT of '/svn/repo/!svn/vcc/default': Could not read chunk delimiter: Secure connection truncated (https://svn.myco.com <https://svn.myco.com/> )
We have tried to nail down why it only fails sometimes, but this is consistent accross networks and different clients etc. Local checkouts via file:/// always works.
After much googling the only thing that helps is setting:
SVNPathAuthz off
which is not good enough security wise for our purposes. I found out about this by reading this:
http://www.sharp-tools.net/archives/000928.html
which is the only reference to this exact issue that I can find. I tried to leave a post on that page to ask exactly how they found out that there was a memory leak but the comment system is broker (server 500 page :(
Another possible workaround would be to set up a cron job like this (at least it *seems* to work for us although I have not tested it long enough and definetly not put it into production... )
* * * * * /etc/init.d/httpd configtest 2>/dev/null && /etc/init.d/httpd graceful #guarantee that every http requset gets a newly started fresh thread
Seems I'm not the only one asking:
http://groups.google.se/group/linux.debian.bugs.dist/browse_thread/thread/d14c54c3b8a56255
http://svn.haxx.se/dev/archive-2008-02/0628.shtml
Anyway it makes sense that http worker threads leak memory and then at some point just die and drop their connections.
How can I verify the memory leaking and supply more concrete info ? ( I'd rather not sit and stare at top and hope to catch defunct processes since the errors occurs say five times per day )
Regards,
Alex B
Legal Disclaimer:
***********************************************************
This e-mail and the information it contains may be privileged and/or confidential.
It is for the intended addressee(s) only.
The unauthorized use, disclosure or copying of this e-mail, or any information it contains, is prohibited.
If you are not an intended recipient, please contact the sender and delete the material from your computer.
***********************************************************
Received on 2008-05-30 09:38:32 CEST