[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: 1.7.0-beta2 this afternoon

From: Greg Stein <gstein_at_gmail.com>
Date: Thu, 21 Jul 2011 08:23:39 -0400

On Thu, Jul 21, 2011 at 08:09, Philip Martin <philip.martin_at_wandisco.com> wrote:
> Greg Stein <gstein_at_gmail.com> writes:
>
>>> The first thing I tried leaks memory:
>>> http://subversion.tigris.org/issues/show_bug.cgi?id=3967
>>
>> Are you suggesting that my fix for 3888 caused this problem, or is
>> incorrect in some way? Or is this a new, distinct, and undiscovered
>> problem that you're bringing up?
>
> Issue 3888 was something to do with file transfers?  I see 3967 with
> empty revisions so it's probably different.

The root problem with 3888 is that N server requests would be queued
up, where N could grow into the tens of thousands. There are various
bits to try and minimize the memory space of each of those requests,
but it still ended up needing a huge/unbounded amount of memory.

The fix was to halt processing of the server response (that indicated
*which* requests to make) when the outstanding request count gets too
high. Once the count drops, then response processing resumes and more
requests are queued.

If the svnrdump process generates a hojillion independent requests,
then it may be seeing a similar problem. Applying throttling similar
to the update process should fix it.

If the process is single-request based (eg. 'svn log' and a million
log entries in a stream), then we'd have to look elsewhere. That would
be an iterpool-like symptom such as we just found in the log
processing.

Cheers,
-g
Received on 2011-07-21 14:24:11 CEST

This is an archived mail posted to the Subversion Dev mailing list.