Re: Avoinding file handle leak using the Python bindings & core.Stream
From: Trent Nelson <trent_at_snakebite.org>
Date: Tue, 17 Apr 2012 13:41:25 -0700
On Apr 16, 2012, at 5:29 AM, Willmer, Alex (PTS) wrote:
Any chance you could post more of your code? I'm interested in your main loop and all Subversion binding calls -- feel free to omit the full-text indexing logic details. If it's company code that you can't share, uh, feel free to accidentally send it to me privately or whip up some pseudo code that shows the binding calls ;-)
Side bar: I've found the ``psutil`` helpful in the past for tracking down file handle leaks:
Can I just clarify you're using the SWIG bindings and not the ctypes-based ones, too?
(I ran into a peculiar memory leak a few months back; I had similar code that essentially analyzed every revision in a repository. So, it's not unfathomable that there could be a leak.)
It's bed time so I'm not going to look at the source right now -- but, if streams are wrapped in the apr_pool weakref black magic, then yeah, .close() could be happening behind the scenes when an object gets garbage collected.
Are you noticing constant memory usage whilst analyzing the repo or does that grow as well?
> 2. Disregarding Python, in the Subversion library is it required that every svn_stream_t created (by eg a call to svn_fs_file_contents) is explicitly closed, or is there some automatic clean-up/closure provided by the pool system?
Again, not looking at the code, I'd be inclined to blame the Python bindings, not the Subversion libraries. I'd wager that Subversion takes care of cleaning itself up when the stream's pool gets destroyed. It's pretty good like that.
This is an archived mail posted to the Subversion Users mailing list.