Re: Potential regression: high server-side memory consumption during import

On Sat, Mar 03, 2018 at 03:10:46PM +0100, Stefan Sperling wrote:
> Regardless, could you try one more test run with this diff against
> trunk to see if it has any impact already?

My new diff doesn't help, it brings the memory problem back.

I have done some more digging with APR POOL DEBUG.
It is the connection pool which is getting too large:

POOL DEBUG: [98085/19516226785696] PCALLOC ( 319481/ 2339944/ 19247334) 0x11bf5c548a00 "transaction" <subversion/libsvn_repos/authz.c:1646> (10974/10974/1)
POOL DEBUG: [98085/19516226785696] PCALLOC ( 319505/ 2340523/ 19247913) 0x11bf5c548a00 "transaction" <subversion/libsvn_repos/authz.c:166>
(10975/10975/1)

The first allocation is the svn_authz_t returned from svn_repos_auth_read3().
The second one is the result of construct_authz_key().

So the problem is that svn_repos_auth_read3() is allocating a new svn_authz_t
in the result_pool. We cannot store this object in the connection pool without
running out of memory with many requests. Which means the patch I already
committed is the "right" fix if we want to keep calling svn_repos_auth_read3()
for every request.

The authz cache already uses a separate pool, the 'authz_pool', to store
a parsed representation of the authz config. This pool seems fine.

Right now I have no idea how to make the current code in trunk any
faster without reverting r1778923.
Received on 2018-03-03 16:27:56 CET