Re: RFV on issue 860 (httpd memory usage)

On Mon, 2002-11-04 at 16:06, Karl Fogel wrote:
> Anyone who's feeling squirrely, please take a look at
>
> http://subversion.tigris.org/issues/show_bug.cgi?id=860
>
> Toward the end, you'll see Ben Collins-Sussman's summary of the
> current situation, about how the bug's severity has been reduced, yet
> there is still some mysterious linearity going on.
>
> I'm going to be looking at this too, but this is one of those
> situations where many eyes might help. Especially those with
> experience tracing Berkeley DB memory usage and/or APR pools.
>

Karl,

        At the end of the issue, it talks about it might be apache or Berkeley
DB. However, at the top it mentions that this happens with ra_dav or
ra_local. So I'm failing to see how it could be apache if it does it
over ra_local.

        If this happens when using ra_local (debugging apache intimidates me...
sorry I'm a wimp), I would take a look at it. Leaking memory is
something I grok well enough, that I feel like I can contribute.
Between strace and LD_PRELOAD (on brk and mmap) it should be possible
(maybe even easy) to find a backtrace of where the memory is being
allocated from then send in backtraces that are the worst offenders.
Maybe I'll finally remember to use that commit access to commit it to
dev/tools too.

        I'm kinda curious if this is a WAD (Works as Designed) issue with
Berkeley DB. You're writting a lot of information to a lot of different
log files that you're not committing. I don't know enough about
Berkeley's infrastructure to comment, but has anybody asked them about
the overhead of having an active transaction that crosses 100-1000 log
files (log files are still 1M right?), and committing it has to write a
compressed version of the file out to the logs right so your talking
between 100-1000 log files (depends on the data and the compression).
So to take up the 28MB difference between 100M and 900M of data. 900M
of random data compresses into ~900 1M log files. So is right about 32K
of overhead per open log file in a transaction assuming no compression.
32K sounds like a reasonable number (pretty high given BDB's embedded
usage, but maybe), so it might be there.

        Maybe modify the log file size on the creation to be 10M and see if the
100M starts acting like 10M that's my bet.

        Maybe Daniel Berlin knows, I'm pretty sure he's the one who set
straight some things for me about BDB on the list in the past.

        Thanks,
                Kirby

> -Karl
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
>

-- 
Real Programmers view electronic multimedia files with a hex editor.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org