[ this should have on-list; forwarded with permission ]
----- Forwarded message from "Kirby C. Bohling" <kbohling@birddog.com> -----
From: "Kirby C. Bohling" <kbohling@birddog.com>
Subject: Re: What is handling it streamily?
To: Greg Stein <gstein@lyra.org>
Date: Mon, 04 Feb 2002 16:24:14 -0600
Greg,
Well, from what I can tell, the pool that is a directory baton is being
over used. I am pretty sure it represents 60-80% of the memory. I know
I saw the directory baton pool up over 40MB, but I believe that was
early in the check out process. I had a patch that showed it quite
clearly, but I did an svn update and now I can't get the code to build
again :-(. I have a diff around to do it pretty easily again. I have
tried to follow the directory baton around, and well it makes my blood
come out my ears... editors, deltas, batons, Oh my!... There are a lot
of functions and function pointers that make it difficult to follow, but
pluggable as hell so it is all good. I think I have a working copy from
a fresh checkout.
It shows all the pools and the sizes. I used tags and file_line from
the pools to determine exactly which pools and where they were created
that the amount of memory is being allocated in. I know that a ton of
memory is consumed creating the entries file over and over again in the
same directory. I haven't looked at the rest of it. The stuff for
files gets pretty large also. The number of allocations over 2K is
small. It would be easy to add statistics for that too.
I have code that will allow you to take a pool and print a message every
time you allocate from this specific pool, set a breakpoint there, and
you can see the call tree of every memory allocation to the pools that
are a pain. You will lose your mind reading all of the allocations
ever. It seems you nickel and dime them to death, and keep a lot of
memory around a ton longer then needed. The lifetimes are all wrong is
my guess, but I can't tell that because there is too much code I don't
know about. I know that Mozilla has a tool named "SpaceTracer" I
believe that tracks memory footprint and lifetime from logs to help you
find objects that are living too long. I spent a ton of time with
OpenStep chasing stuff like this down in a former life, it is a lot more
fun when it isn't your job :-)
If you want pathes and/or reports on this stuff let me know, I will send
it. Some of it requires patching to apr, and some of it is written in
such a way that you have to have SVN_POOL_DEBUG on, but it is all
fixable to be relatively sane. Some scripting/gdb/libiberty, and I can
probably nail down where every last allocation comes from on a
particuarly pool if that is useful. I don't know that the apr people
will like the apr changes or accept them, which makes a lot of the stuff
less useful. It's why I haven't submitted it to the list yet, I thought
I might find a way around the changes to apr. It helps a lot in finding
the pools.
Kirby
----- End forwarded message -----
--
Greg Stein, http://www.lyra.org/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Sat Oct 21 14:37:04 2006