[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Python bindings: Use of svn_swig_py_make_stream() - avoiding leaking fds

From: Piper, Nick <nick.piper_at_logica.com>
Date: Sat, 7 May 2011 05:00:05 +0000

Dear svn-devs,

I'm trying to work through some repository corruption problems we're having, and one problem I've noticed during the investigation is that our Python application (running in mod_wsgi in Apache httpd) appears to be keeping (deleted) files open in /tmp.

I've narrowed this down to our use of delta.svn_txdelta_send_stream(f, handler, baton).

In Python, f is a "file" instance, but that will be converted to a svn_stream_t by swig (by svn_swig_py_make_stream()).

That file then goes out of scope in Python, but the file descriptor is never closed. I don't see where we should be calling svn_stream_close() in order to do that (and call Py_DECREF(py_io)), because the svn_stream_t is never exposed back to Python.

I'm using both svn 1.6.9 and 1.6.12 (all Linux, CentOS/Debian.)

a) Is there a better way for us to be 'replacing' an svn file with new content, but that still does not require us to have the whole new content in memory at once? The way we're doing now, we've been doing since http://svn.apache.org/viewvc?view=revision&revision=854879 (probably) but maybe it only started exhibiting this behavior at http://svn.apache.org/viewvc?view=revision&revision=861447. The putfile.py example in svn does it 'all contents in memory' using delta.svn_txdelta_send_string(). Maybe when I've solved this problem I can fix the "it would be nice" TODO item in that example :-)

b) Are any of the swig functions which convert a python io object into a svn_stream_t 'safe' for long-running applications due to this problem, or (probably) am I missing a step in our code?

I've attached a script which reproduces the problem in a minimal way.

Thank you for any help,

 Nick

-- 
Nick Piper MEng MIET RHCE| #define Joint Lead Architect
250 Brook Drive, Green Park, Reading RG2 6UA | United Kingdom
nick.piper_at_logica.com | www.logica.com 
Logica UK Limited, registered in England & Wales (registered number 947968)
Registered Office: 250 Brook Drive, Green Park, Reading RG2 6UA, United Kingdom 
Think green - keep it on the screen.
This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you. 
Received on 2011-05-07 07:00:51 CEST

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.