On Mon, Nov 8, 2010 at 1:23 PM, Paul Burba <ptburba_at_gmail.com> wrote:
> On Mon, Nov 8, 2010 at 10:35 AM, Paul Burba <ptburba_at_gmail.com> wrote:
>> On Mon, Nov 8, 2010 at 9:13 AM, Paul Burba <ptburba_at_gmail.com> wrote:
>>> On Thu, Nov 4, 2010 at 8:59 PM, Chris Tashjian <ctash_at_thepond.com> wrote:
>>>> I posted this on the users list, but I'm confident that this is a bug.
>>>>
>>>> Background:
>>>> For a while now (off and on for over a year, but more frequently in the last
>>>> 6+ months) we've been having problems with svn "crashing", yet there's no
>>>> error in the log file. In talking to someone the users list it sounds like
>>>> svn is actually just hanging. Clients get the following response:
>>>>
>>>> svn: Can't connect to host 'svn': No connection could be made
>>>> because the target machine actively refused it.
>>>>
>>>> Our repository has 129K revisions. The format is "4 layout linear", it was
>>>> created with svnadmin 1.4.x and has since had "svnadmin upgrade" run both in
>>>> 1.5 and 1.6. We're currently running SlikSVN 1.6.13 (Win32), but I have
>>>> previously had this problem dating back to versions of 1.5, both stock and
>>>> from CollabNet. The issue now happens numerous times per day and it looks
>>>> like I've discovered why....
>>>>
>>>>
>>>> As a test I ran "svn blame -g" on a file with a bunch of revisions and
>>>> watched memory usage on the server spin up to 2GB.
>>>
>>> Chris,
>>>
>>> By a "bunch of revisions" what exactly do you mean? Many revisions
>>> which were the result of a merge? Or simply many changes made
>>> directly to the file (not the result of a merge)?
>>>
>>>> Paul - I'll see if I can get a test repo up with the error. In the
>>>> meantime, would a copy of the svn:mergeinfo help?
>>>
>>> Any luck?
>>
>> Chris,
>>
>> Don't sweat the replication effort too much, I think I'm on the trail
>> of this problem. Using a copy of the old (pre-ASF) Subversion
>> repository I'm seeing unexpectedly high memory use when using log -g
>> on a file with a "lot" of merge history:
>>
>> 1.6.13.client>svn blame ^/trunk/subversion/tests/cmdline/merge_tests.py
>> 25 MB Peak Working Set Memory svnserve.exe
>>
>> 1.6.13.client>svn blame ^/trunk/subversion/tests/cmdline/merge_tests.py -g
>> 291 MB Peak Working Set Memory svnserve.exe
> ^^^
> That should have been 691 MB! Sorry!
>
>>
>> More soon...
Hi Chris,
Ok, I think I got it. Switching to a Subversion trunk_at_1032651 (debug)
build and using this repository as a test*:
http://svn.collab.net/tmp/subversion-data-backup/subversion-history-20091115.tar.bz2
The peak working set memory of
'svn blame ^/trunk/subversion/tests/cmdline/merge_tests.py' was 21 MB
'svn blame ^/trunk/subversion/tests/cmdline/merge_tests.py -g' was 574 MB
That's comparable to what you saw with 1.6.13.
Applying some standard pool-fu to
libsvn_repos/rev_hunt.c:(find_merged_revisions|find_interesting_revisions),
see attached patch, and things look a *lot* better:
The peak working set memory of
'svn blame ^/trunk/subversion/tests/cmdline/merge_tests.py' stays at 21 MB
'svn blame ^/trunk/subversion/tests/cmdline/merge_tests.py -g'
drops to 71 MB
A tidy 88% reduction in peak memory usage!
Running the test suite on this patch. Assuming no problems I will be
committing and nominating for backport to 1.6.x.
log:
[[[
Fix blame -g server-side memory leaks.
See http://svn.haxx.se/dev/archive-2010-11/0102.shtml
* subversion/libsvn_repos/rev_hunt.c
(find_interesting_revisions): Implement result/scratch two-pool paradigm.
Don't needlessly allocate path_revision structures until we are sure
we are going to keep it. Rename local variable "iter_pool" to the
de facto standard "iterpool".
(find_merged_revisions): Use a separate iterpool for each nested loop.
Update call to find_interesting_revisions, passing, you guessed it, an
iterpool. Rename local variable "iter_pool" to the de facto standard
"iterpool".
]]]
Paul
* This is the old Tigris Subversion repository, see
http://svn.apache.org/repos/asf/subversion/README.
Received on 2010-11-09 02:17:55 CET