On 29.12.2010 08:23, Blair Zajac wrote:
> On 12/28/2010 6:28 PM, Stefan Fuhrmann wrote:
>> On 29.12.2010 01:33, Stefan Fuhrmann wrote:
>>> Hi devs,
>>>
>>> I would really like the core improvement of my
>>> work on SVN performance to become part of
>>> 1.7: the in-process fulltext cache.
>>>
>>> While the risk is low (we already offer something
>>> similar based on memcached), the caching code
>>> is so fundamental to most other pending changes,
>>> that I want to hear at least a second opinion:
>>>
>>> ^/subversion/branches/integrate-cache-membuffer
>>>
>>> Take this as an opportunity for a last nightly deed
>> *k*nightly (which stresses even more that _nightly_
>> is exactly what we probably don't want)
>>> (Ny!) in 2010 or a new year's resolution.
>
> I'll take a look at it when I have a chance, but my first question is,
> does it work in a long running (months) persistent multithreaded
> process that LRU caches svn_fs_t, svn_repos_t and other objects for
> long periods of time? I have a custom svn server that exposes the
> svn_fs.h API through an Ice layer (see www.zeroc.com).
That should not be a problem: The cache uses a
separate root pool and allocator object. It also
frequently cleans sub-pools used for temporary data.
Assuming that you clean the pools that receive the
FSFS content (otherwise your server would run OOM
quickly with the current code), you should be fine.
Although my longest run so far was 24 hours, in
combination with further optimizations it served
0.5PB (20 billion files) from cache in a load test
and memory consumption oscillated in a very narrow
range (<1%).
BTW, the caching code is used by the FSFS back-
end only. So, the impact on your custom svn server
may be zero.
-- Stefan^2.
Received on 2010-12-29 20:18:00 CET