On Mon, Jun 8, 2009 at 11:02 AM, Justin Johnson <ajustinjohnson_at_gmail.com>wrote:
> On Tue, Jun 2, 2009 at 12:31 PM, Mark Phippard <markphip_at_gmail.com> wrote:
>
>> On Tue, Jun 2, 2009 at 1:24 PM, Bolstridge,
>> Andrew<andy.bolstridge_at_intergraph.com> wrote:
>> >> -----Original Message-----
>> >> From: B Smith-Mannschott [mailto:bsmith.occs_at_gmail.com]
>> >> Sent: Friday, May 29, 2009 11:27 AM
>> >> To: Bolstridge, Andrew
>> >> Cc: users_at_subversion.tigris.org
>> >> Subject: Re: 1.6.2 poor commit performance with large data
>> >>
>> > [snip]
>> >
>> >> This sounds like the same problem as described in [this thread][1]
>> >> from a few days ago. Said thread links to [Issue 1964][2] which looks
>> >> like the same problem.
>> >>
>> >> [1]:
>> >>
>> http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessage
>> >> Id=2352720
>> >> [2]: http://subversion.tigris.org/issues/show_bug.cgi?id=1964
>> >>
>> >> So far, evidence suggests that this issue crops up only under Windows.
>> >> I haven't seen anything suggesting *why* this is the case, so any
>> >> information you can provide is likely to be helpful.
>> >>
>> >
>> >
>> > I've had a look at this running again today. It seems to be throttled in
>> the System process (not any of the sub-processes, so it must be a driver).
>> On my quad-core box System.exe hits 25% CPU and stays there.
>> > There is a lot of page faults (after transferring 73Mb, I've got 74
>> million page faults with a delta of roughly 1000 per second).
>> > Memory usage steadily (but very slowly) increases once the transfer has
>> started to stall, currently showing 169Mb/164Mb in taskmanager.
>> >
>> > and then.... I had a brainwave. I turned off the virus checker and
>> suddenly things went back to normal. (I use Trend which isn't quite so bad).
>> The memory usage is still increasing at a rate of 1Mb per second but the
>> page fault delta for System process has dropped to 0.
>> >
>> >
>> > In the end, transferring 9500 mostly cpp source-code files (187Mb) used
>> peak memory of 1.5Gb RAM.
>> >
>> > If anyone wants to check the 'leak', I made a heap comparison showing
>> the allocations. I did this using TortoiseSVN (as that has some symbols),
>> unfortunately it doesn't have enough symbols for the svn libraries or it'd
>> be more useful. If anyone has built SVN for Windows, and will create some
>> .pdb symbols, then I'd be happy to re-run my commits and see if I can find
>> out more about this issue.
>> >
>> > It appears that the call to libapr_tsvn's apr_allocator_max_free_set is
>> doing the damage, after transferring 130Mb (roughly 7000 files), it has
>> 'leaked' 749Mb. I can’t see what has been calling it without some effort
>> (and symbols) but maybe someone more knowledgeable of SVN source can offer
>> some suggestions. It appears to mostly allocate 8k at a time, but
>> occasionally allocates 16k (if that helps any).
>> >
>> > Be aware I've cross-posted this to the TortoiseSVN list too.
>>
>> The leak has been discovered and fixed and will be in 1.6.3. This
>> also revealed that the way temp files are created is a cause of
>> slowness. A fix for that is being looked at and will hopefully be in
>> 1.6.3 as well.
>>
>
> Is this issue only on the client side? Any memory leaks on the server?
>
> When is 1.6.3 supposed to be released?
>
> Thanks.
> Justin
>
>
>
For anyone who is interested, I tried adding the source code for the linux
2.6.29.4 kernel to a working copy and committing. From Windows (1.6.2
command line) the commit slowed down rather quickly and consumed all my
memory. It took so long (5 hours +) I eventually just killed the process.
I then tried the same thing from a Solaris (1.6.2) working copy. It went
much faster, but still slowed down after a while and eventually failed with
the following error.
Out of memory - terminating application.
Abort
So the problem is on Windows and UNIX, but I'm still waiting to hear if the
leak affects the 1.6.2 server at all, and also when the 1.6.3 release will
be out.
Justin
------------------------------------------------------
http://tortoisesvn.tigris.org/ds/viewMessage.do?dsForumId=4061&dsMessageId=2360414
To unsubscribe from this discussion, e-mail: [users-unsubscribe_at_tortoisesvn.tigris.org].
Received on 2009-06-08 20:57:47 CEST