Re: Db4 shared mem, ENOMEM on large import

On Fri, Mar 08, 2002 at 03:55:30PM -0500, Ben Collins wrote:
>...
> I don't think we are leaking locks, but we may be doing too much work
> (I'm not familiar with the internals of the locking scheme in bdb) at
> once.

This is exactly what is happening. From our discussion about revamping the
Node IDs earlier today, we talked through a lot of issues. The big one,
which you're running into is that the entire commit, stabilization, *and*
deltification all occurs within one mother lock.

Besides the lock count problem, this just sucks for scalability, and for
sharing the database among multiple users. We're just holding the top lock
for way too long (relatively speaking).

Our solution is to switch to the new Node ID form, to completely remove the
entire stabilization pass (no more immutable flags in the nodes, and no
renumbing of nodes), and to move the deltification to a operate in a
post-commit fashion, outside of the lock.

This means, we'll have one teeny lock as we insert a row into the
repository-version table. Then we can start deltifying, but that process can
do a lot of its work outside of any serious lock.

We isolated a number of steps in the sequence. I'll detail that tomorrow
(off to bed now), and I have about a half-dozen issues to enter too.

Cheers,
-g

-- 
Greg Stein, http://www.lyra.org/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org