On 07.03.2011 13:11, Philip Martin wrote:
> stefan2_at_apache.org writes:
>> Author: stefan2
>> Date: Sat Mar 5 21:18:33 2011
>> New Revision: 1078357
>> Modified: subversion/trunk/subversion/svnadmin/main.c
>> URL: http://svn.apache.org/viewvc/subversion/trunk/subversion/svnadmin/main.c?rev=1078357&r1=1078356&r2=1078357&view=diff
>> ==============================================================================
>> --- subversion/trunk/subversion/svnadmin/main.c (original)
>> +++ subversion/trunk/subversion/svnadmin/main.c Sat Mar 5 21:18:33 2011
>> @@ -290,6 +290,11 @@ static const apr_getopt_option_t options
>> N_("use format compatible with Subversion versions\n"
>> " earlier than 1.7")},
>>
>> + {"memory-cache-size", 'M', 1,
>> + N_("size of the extra in-memory cache in MB used to\n"
>> + " minimize redundant operations. Default: 256.\n"
>> + " [used for FSFS repositories only]")},
>> +
>> {NULL}
>> };
>> @@ -1650,6 +1656,7 @@ main(int argc, const char *argv[])
>> /* Initialize opt_state. */
>> opt_state.start_revision.kind = svn_opt_revision_unspecified;
>> opt_state.end_revision.kind = svn_opt_revision_unspecified;
>> + opt_state.memory_cache_size = 0x10000000; /* 256 MB */
> 256MB is a magic number. How was its value determined?
It has been the "sweet spot": Significant performance gains
even for larger repositories but the initialization overhead
is still small even for smaller repositories.
> It causes the regression tests to fail on machines with 2GB of memory
> when the tests are run in parallel, as they cannot allocate 10*256MB.
I wasn't aware that anybody would do that.
> There appears to be a significant overhead setting up the cache.
> Setting the magic number to 16MB makes the tests noticeably faster,
> setting it to 1MB makes the tests faster still.
The fix for overcommitment makes memory allocation slow
(~4GB / sec, YMMV). Also, without r1078990, I/O dominated
the runtime of svnadmin verify & friends rendering the
txdelta cache more or less ineffective.
One point that has been missed in this discussion so far,
is the fact that 1.6 FSFS already uses very large caches.
For instance, it will cache up to 8k directories which will
require several MBs once that cache fills up. With the
membuffer cache, the limit on memory consumption is
quite solid:
svnadmin verify -q $tsvn -M256 : 263MB tops, 56s real (old default)
svnadmin verify -q $tsvn -M16 : 24MB tops, 64s real (new default)
svnadmin verify -q $tsvn -M1 : 8MB tops, 157s real
svnadmin verify -q $tsvn -M0 : 242MB tops, 104s real
svnadmin-1.6 verify -q $tsvn -M0 : 250MB tops, 213s real
So, the point here would be making "make check" faster
but the svn tools themselves should be fine with the current
defaults for real-world repositories.
-- Stefan^2.
Received on 2011-03-08 20:39:14 CET