On Thu, 2011-04-14 at 23:22 +0200, Johan Corveleyn wrote:
> Hi Neels,
>
> Great work, and good to see those numbers. But ... hold on a minute
> before jumping to conclusions. IIUC you only tested one type of
> platform: *nix with local disk (on a (powerful) VM, and on a (slightly
> less powerful) "real machine"). It gives a good indication, but it's
> definitely not enough to declare trunk release worthy
> performance-wise.
I'm aware of that and am usually very careful to place "I'd say" and
"IMHO" all around my humble "conclusions" ;)
As-is, my scripts definitely won't run on Windows. The core is python,
but there's a thin layer of bash around it that calls the python with
the desired settings and filenames, so the bash stuff needs to be looked
at and translated to BAT ...or python probably.
It could work on OSX without changes, AFAIK.
Frankly, I won't personally go into testing on Windows or a WC on an NFS
mount, because, *IMHO*, both Win and WC-on-NFS are misguided, and I
personally can't be bothered. This is where people using those things
and whose jobs depend on svn 1.7 doing well on them should get involved,
as far as I'm concerned.
Johan is right to highlight the limitations. And I can add a large
portion of limitations inherent to benchmarks like these (hardware
particulars, time measurement inaccuracies, simulation validity,
statistics, interpretation, selective perception, chance,... ). Only the
actual release 1.7.0 will bring all the real issues to the surface.
And yet, the conclusion I find from my past and present test runs is:
the performance I have seen in my setup a while back has definitely been
a show-stopper. And now I don't see those particular show-stoppers
anymore. So, yes, I should probably be more clear instead of saying
"release-worthy" -- I meant it rather negatively defined, as in: my
personal perf show-stoppers have vanished. (Thanks for poking)
So, does anyone do Windows? Do you speak bash, python and BAT? Want to
know about perf on your setup? I'll gladly send you my scripts and
explain anything you want to know about it.
Oh, and about the time factors getting much better on the VM -- that's
not just because the VM is faster than my home machines, since then
1.6.x would also be faster on the VM, resulting in similar time factors
as on other hardware. I don't know in detail why we're seeing this, but
it must be something like, maybe SQlite has some magic that enhances
performance when -- I don't know -- the CPU bus is very wide, or disk
cache is fast, or something like that, boosting fast hardware even more.
It must be something that svn 1.6 can't benefit from.
Enough talk :)
~Neels
>
> For all we know there could be a massive regression on Windows or
> MacOS, or on NTFS specifically (like the locking in WC-1, which was
> orders of magnitude slower on Windows/NTFS than on Linux), or, as
> already hinted a few times by Philip, on network-mounted filesystems:
> NFS, CIFS, ...
>
> So, to get a better picture, your test-suite should be run on a
> variety of platforms (others can help, I guess, like with Mark's
> perf-tests). Can your test-suite be run "as is" by others on Windows
> for example? Maybe you could already run it yourself on an NFS-mounted
> volume?
>
> I think we need to at least test [ *nix | Windows | MacOS ] x [ local
> disk | remote disk ] (ok maybe *nix is too coarse-grained, as is local
> disk etc..., but you get the picture).
>
> Cheers,
Received on 2011-04-15 01:25:45 CEST