Re: File descriptor leak of rep-cache.db in 1.6.x

On 3 Jun 2010, at 18:39, Hyrum K. Wright wrote:

> On Thu, Jun 3, 2010 at 11:26 AM, Dan Villiom Podlaski Christiansen <danchr_at_gmail.com> wrote:
> I just switched my checkout to that branch, compiled and ran the tests. And it worked! Great!
>
> Is there any particular reason why it hasn't been merged into the 1.6.x branch yet?
>
> It just needs one more vote, as per STATUS:
> http://svn.apache.org/repos/asf/subversion/branches/1.6.x/STATUS
>
> Lobbying here or on IRC for that vote will improve it's chances of being cast. (But I suspect that it will happen before we get to 1.6.12.)

Having this fix in 1.6.12 would be great. How far off is 1.6.12?

> > If converting a repository is a bit like "svnadmin dump" then it may
> > not be SWIG, you might be seeing issue 3593:
> >
> > http://subversion.tigris.org/issues/show_bug.cgi?id=3593
>
> That's interesting. I'm currently doing a dump-and-load of the cvs2svn repository. (For some reason, this is obnoxiously slow…) I'll give it a try once it completes and report back.

FWIW this BDB slowness appeared exclusive to my Mac; the dump-and-load went much faster on my FreeBSD VPS.

> For what it's worth, we also get massive leaks with the SWIG bindings when cloning/converting over the network. I'm sure this isn't the only cause of those leaks :)
>
> :( For which flavor of the swig bindings are you seeing problems?

We're using the Python SWIG bindings. Some statistics:

BDB repository w. Subvertpy:
Memory: 37.7 MB virtual private
Running time: 6m38s CPU

FSFS repository w. Subvertpy:
Memory: 417.6 MB virtual private
Running time: 6m36s CPU

BDB repository w. SWIG:
Memory: 413.3 MB virtual private
Running time: 3m03s CPU

FSFS repository w. SWIG:
Memory: 408.9 MB virtual private
Running time: 3m04s CPU

Based on this I would say that BDB repositories appear to not leak much when using Subvertpy. Everything else leaks quite a lot. Granted, there is a significant CPU overhead in Subvertpy, but I believe its much cleaner API and better memory handling compensate for that. Besides, many people using hgsubversion will have their conversions I/O bound rather CPU bound.

--
Dan Villiom Podlaski Christiansen
danchr_at_gmail.com