On 09.03.2011 11:56, Philip Martin wrote:
> Stefan Fuhrmann<eqfox_at_web.de> writes:
>
>> On 08.03.2011 14:46, Philip Martin wrote:
>>> stefan2_at_apache.org writes:
>>>
>>>> Author: stefan2
>>>> Date: Mon Mar 7 22:57:04 2011
>>>> New Revision: 1079008
>>>>
>>>> URL: http://svn.apache.org/viewvc?rev=1079008&view=rev
>>>> Log:
>>>> Set FSFS cache default size to 16 MB. This is the same default as
>>>> for everybody else, namely the server processes. Thus, it should
>>>> be reasonable value on the same machines.
>>> 16MB may be "reasonable" for normal usage but it still has a significant
>>> effect on the testsuite. The testsuite is an unusual access pattern, it
>>> runs thousands of commands on small repositories. Using a 1MB cache I
>>> can run the testsuite for ra_local/FSFS in 10m15s, with 16MB cache that
>>> increases to 12m30s, a 20% increase. There is a similar increase when
>>> testing ra_svn/FSFS using the default Linux svnserve (a threaded
>>> svnserve has a shared cache so isn't affected).
>> The patch attached fixes the performance penalty
>> for "make check" by disabling membuffer caches
>> over ra_local by default. Since svn_ra_initialize is
>> being called early in main(), the setting can later
>> be overwritten by settings from e.g. a config file.
>>
>> Could you verify that / whether this fixes your
>> performance issue?
> It goes some way, but it's not as good as setting the cache size to 1MB
> since it doesn't affect the svnadmin dump/load commands used by most of
> the tests.
I think we should commit that change, then. But I'm not
perfectly sure that the ra_loader initialization code it is
a good place to put it. It seems to be one of the central
places where it could be done consistently.
>>> svnserve and svnadmin both have a -M option, was there any discussion
>>> about using a single letter option for this feature?
>> No.
>>> This could be used
>>> by the testuite to restrict the cache size, but for ra_local/FSFS
>>> testing there is no way to restrict the svn client.
>> As already shown in a different thread, the memory
>> consumption should actually go down for any non-
>> trivial repository.
>>> Should we add the -M option to the svn client?
>> Due to its limited applicability (FSFS via file:/// only),
> There may well be large numbers of people using that for private
> repositories.
With the cache statistics feature we talked about lately
on IRC, we should get reliable data whether ra_local
will benefit from it at all. Having thought about that for
some time now, it seems that even diff and annotate
might not benefit.
The TSVN repository browser, in fact, might be one
of the few applications that causes significant numbers
of cache hits.
> As I undertand it cache initialisation includes setting the memory to
> zero, and that this is partly to ensure that all the allocated memory is
> really available to the process on a Linux system with overcommit
> enabled.
Exactly. I wanted to prevent a behavior that looks like
a memory leak. Non-threaded servers should now
be more likely to detect the OOM condition and fall
back to the hopefully smaller 1.6 caching scheme.
> If the cache had a more lightweight initialisation then a
> cache size of 16MB would probably not be a problem.
The index part needs to be initialized but covers only
1/16th of the whole cache memory. So, yes, no zeroing
the data memory will bring performance back.
> Was there any
> discussion about this overcommit behaviour? Linux overcommit is
> configurable, why should Subversion override this?
It would a service to all these LAMP V-server users
that might not be aware about the overcommit feature
at all.
> It doesn't apply to
> all the other memory allocation in Subversion, so what guarantee are you
> trying to provide and what are you actually providing?
Good point. You got me convinced: the 1.6 caches
will grow slowly while getting filled and without zeroing,
the membuffer cache will apparently do the same.
Committed in r1081082.
-- Stefan^2.
Received on 2011-03-13 12:44:55 CET