new mem-management for libsvn_fs?

I've been looking at issue #860, whereby we're still seeing waaaaay
too much memory used by libsvn_fs, even using today's latest code.
This is a serious scalability problem. After many discussions with
brane and cmpilato, I'd like to propose a new memory-management
philosophy for libvn_fs.

The status quo isn't so hot. Here's how pools currently work:

* caller passes a pool to the fs.

* the pool is passed to retry_txn(), which uses it to create a trail.

* trail->pool becomes a subpool of the caller's main pool.

* the fs routines allocate both scratchwork and return-data in
  trail->pool, and therefore this subpool is never allowed to be
  freed.

==> lots of wasted memory.

In the proposed scheme, we require the separation of scratch-data from
return-data:

* caller passes a pool to the fs.

* the pool is passed to retry_txn(), which uses it to create a trail.

* trail->pool becomes a subpool of the caller's main pool.

* every txn_body() routine changes signature:

     * txn_body(baton, trail) ==> txn_body(baton, trail, returnpool)

     * txn_body routines continue to let fs routines use trail->pool
       for both scratchwork and return-data as usual, but before
       exiting, makes *copies* of important data into the returnpool.

     * retry_txn() simply passes the main caller pool as the
       'returnpool' to the txn_body routine.

* commit_txn() destroys trail->pool upon exit.

==> only return-data is preserved; all scratchwork is freed.

The real work here is going through every single txn_body_ routine,
and teaching it to make 'deep' copies of return-data into the return
pool. Before I start this, I want Old Hundred Eyes to agree that this
is a good approach. :-) cmpilato and I think it's the way to go.

Feedback?

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Thu Oct 31 23:22:38 2002