On Sat, Jun 21, 2014 at 5:18 PM, Branko Čibej <brane_at_wandisco.com> wrote:
> On 20.06.2014 11:22, Ivan Zhakov wrote:
>
> On 19 June 2014 17:06, Stefan Fuhrmann <stefan.fuhrmann_at_wandisco.com> <stefan.fuhrmann_at_wandisco.com> wrote:
>
> Turn out that the ruby repo is something special
> in that it has very deep histories of relatively few,
> very small files combined with one huge changelog
> file (the latter taking up ~75% of the repo). See
> below for details.
>
> Also, please note that your exports contained
>
> 500000 files. Using 16MB of cache with that
>
> project size *may* not be an adequate setup.
> Upping that to insane 256MB (roughly what 1.6
> would use anyway), gives much better numbers.
> However, there is hardly a difference between
> f6 and f7 in these runs.
>
>
I made this statement before Ivan disclosed what commands he actually ran.
Having 256MB of cache is at least worth a try when dealing with 4GB
checkouts.
I provided the "all defaults" setting as well, so everyone can see how
caching
inside SVN *does* speed things up - an effect that Ivan adamantly denied
exists.
> Here my measurements with svn: under Linux:
>
>
> There is still misleading information about real fsfs7 performance:
> 1. You're comparing fsfs7 packed vs fsfs6 packed and do not provide
> data for fsfs6/fsfs7 unpacked. I already demonstrated you that fsfs6
> unpacked (default) is dramatically faster with defaults options.
>
>
As you can see from the numbers I posted above, packed f7
is not worse than packed f6 even with default settings. Hence,
your statement of "relies on huge caches" has been demonstrated
to be false.
It is true however, that f7 will use caches more eagerly and its
design makes it benefit from moderate cache sizes (.1-1G) more
than f6. Actual gains depend on circumstances, of course.
> 2. You're still testing svn:// protocol only. And you even don't
> bother to test http:// protocol, while I demonstrated you 10 times
> performance degradation on the same test data.
>
> Setting up the HTTP test simply takes me a while. But I will
do it. In a real-world setup, HTTP should deliver data twice
as fast as svn: for exports from cold disks, simply because it
issues concurrent requests and keeps more heads busy.
Also I don't see anything special with repositories with deep
> histories: that's pretty typical for source code.
>
>
> I have to agree with Ivan on the topic of comparing performance
> measurements. Let's try to get rid of observer bias here, please.
>
It's not so much about bias as it is about the time required to
do it. Ivan is using SSDs, I'm using a spinning disk for the tests.
Also, I decided on Friday that I should spend time on understanding
and fixing the issues rather than waiting for even more tests to
complete.
I turned out that the cause of the packed ./. non-packed regression
was a simple change in the 1.9 repo defaults. With r1604414,
packed repositories are now faster than non-packed on everything
with higher I/O latency than a RAM disk (they become CPU bound
on RAM disks). With r1604421, non-packed f7 is no longer more
I/O intensive than non-packed f6.
So, the root causes that we identified for the performance problems
have been addressed now and (re-)testing makes sense. However,
during my root cause analysis, I realized how "naturally grown",
imported and copied repos are very different in their I/O characteristics.
I'll write a post on that shortly.
FWIW. below there are the results from what I did at the office
on Friday. Non-packed repos are entirely I/O bound on spinning
disks with caching hardly making a difference. file:// is slightly
faster than svn:// do to missing protocol overhead. Log is faster
on non-packed repos due to the issue fixed in r1604414.
-- Stefan^2.
time svn-bench null-export $ruby/trunk
F7 non-packed 287.834s, 256MB cache -c 0, svn://
F6 non-packed 244.710s, 256MB cache -c 0, svn://
F7 non-packed 290.476s, all defaults, svn://
F6 non-packed 251.506s, all defaults, svn://
F7 non-packed 287.806s, all defaults file://
F6 non-packed 244.394s, all defaults file://
F7 packed 14.573s, all defaults file://
F6 packed 15.014s, all defaults file://
time svn-bench null-log $ruby/
F6 non-packed 19.615s, all defaults, svn://
F7 non-packed 20.223s, all defaults, svn://
F6 non-packed 17.358s, all defaults file://
F7 non-packed 19.148s, all defaults file://
F6 packed 59.090s, all defaults file://
F7 packed 56.439s, all defaults file://
Received on 2014-06-22 09:52:07 CEST