[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Random serf checkout failures

From: C. Michael Pilato <cmpilato_at_collab.net>
Date: Mon, 5 Nov 2012 13:33:40 -0500

On 11/05/2012 11:49 AM, Lieven Govaerts wrote:
> On Mon, Nov 5, 2012 at 12:41 PM, Lieven Govaerts <lgo_at_mobsol.be> wrote:
>> On Mon, Nov 5, 2012 at 11:33 AM, Philip Martin
>> <philip.martin_at_wandisco.com> wrote:
>>> Philip Martin <philip_at_codematters.co.uk> writes:
>>>
>>>> I've been experimenting checking out subversion/tags from my local
>>>> mirror of the Subversion repository using a client and mod_dav_svn from
>>>> trunk. I'm getting several different errors seemingly at random, I
>>>> suspect they may be related to some common underlying problem.
>>>
>>> I can reproduce this using another dataset that should easier for others
>>> to reproduce: import trunk 150 times making 150 random file changes each
>>> time.
>>>
>>> svn export http://svn.apache.org/repos/asf/subversion/trunk trunk
>>> svnadmin create repo
>>> svn mkdir -mm file://`pwd`/repo/tags
>>> for t in `seq 1 150` ; do
>>> for f in `find trunk -type f | sort -R | head -150` ; do
>>> echo xx$t >> $f
>>> done
>>> svn import -mm trunk file://`pwd`/repo/tags/$t
>>> done
>>>
>>
>> Alright thanks, I'll try this at the hackathon this afternoon.
>>
>
> My current test with svn trunk & apache+mod_dav_svn 1.6.12 does not
> reproduce this issue.
>
> What I am seeing is that svn is using a lot of memory, up to 1GB.
> Currently looking at the root cause there.
>
> First clue: many directories are not closed until the call to
> close_all_dirs in libsvn_ra_serf/update.c:finish_report. This means
> that for the most part of the checkout, the associated info object is
> kept in memory.

My debugging indicates that when close_all_dirs() is called, there are a
slew of unclosed directories which each have a non-zero ref_count. Save for
the root directory of the edit (/tags), fetch_props == TRUE,
propfind_handler->done == TRUE, and tag_closed == TRUE for all open child
directories. So I'm looking around at how file references are managed to
see if anything has leaked. I might have inadvertently botched something
when adding the pull-contents-from-the-wc-cache logic.

-- 
C. Michael Pilato <cmpilato_at_collab.net>
CollabNet   <>   www.collab.net   <>   Enterprise Cloud Development

Received on 2012-11-05 19:34:20 CET

This is an archived mail posted to the Subversion Dev mailing list.