On Thu, Jul 21, 2011 at 08:09, Philip Martin <philip.martin_at_wandisco.com> wrote:
> Greg Stein <gstein_at_gmail.com> writes:
>
>>> The first thing I tried leaks memory:
>>> http://subversion.tigris.org/issues/show_bug.cgi?id=3967
>>
>> Are you suggesting that my fix for 3888 caused this problem, or is
>> incorrect in some way? Or is this a new, distinct, and undiscovered
>> problem that you're bringing up?
>
> Issue 3888 was something to do with file transfers? I see 3967 with
> empty revisions so it's probably different.
The root problem with 3888 is that N server requests would be queued
up, where N could grow into the tens of thousands. There are various
bits to try and minimize the memory space of each of those requests,
but it still ended up needing a huge/unbounded amount of memory.
The fix was to halt processing of the server response (that indicated
*which* requests to make) when the outstanding request count gets too
high. Once the count drops, then response processing resumes and more
requests are queued.
If the svnrdump process generates a hojillion independent requests,
then it may be seeing a similar problem. Applying throttling similar
to the update process should fix it.
If the process is single-request based (eg. 'svn log' and a million
log entries in a stream), then we'd have to look elsewhere. That would
be an iterpool-like symptom such as we just found in the log
processing.
Cheers,
-g
Received on 2011-07-21 14:24:11 CEST