On Fri, Mar 25, 2011 at 1:33 PM, Mark Phippard <markphip_at_gmail.com> wrote:
> I have been working on a framework for writing tests to record
> performance. I have something good enough to share:
>
> https://ctf.open.collab.net/sf/projects/csvn
>
> It is pretty easy to add new tests if you have ideas on more tests you
> think we should add. I think I have pretty good coverage of the major
> functions. The wiki on the site I linked to above has details on how
> I have constructed the current tests. I am going to put out a call to
> users for feedback and try to get more people to run the tests and
> record results.
>
> I am not claiming these are anything definitive or even that we will
> use them to help us make the release decision, but I think it is a
> start on coming up with some reproducible tests that people can run
> easily. If after people look at and run the tests they think they are
> useful or can be tweaked to be useful, then great. If not, then at
> least I got to write some code for a change :)
>
> The tests are written in Java because that is what I know and it gives
> me good cross platform coverage. However, the Java just drives the
> command line so all you need to do is have the svn command line in
> your PATH and that is what it uses for all the work.
These tests are showing some interesting results.
* There are a number of cases where 1.7 shows large performance gains
over 1.6. For example, a WC with a lot of folders on Windows:
https://ctf.open.collab.net/sf/wiki/do/viewPage/projects.csvn/wiki/FolderTests
* Checkout seems to be the biggest remaining hotspot where 1.7 tends
to be slower, although commit seems to still be slower too.
* Delete and move are slower than I would have expected. Is this
because we can actually delete the folder now, as opposed to waiting
until commit time? I actually thought that made would make it faster
as I thought the folder used to get deleted and then put back (maybe
just in IDE's).
* There is something seriously bad going on with Anti-Virus on my
Windows laptop (Symantec Endpoint Protection). I noticed the times
for commit were way out of whack so I configured my A/V to ignore the
folder where the benchmarks were running. When I ran the tests again,
the performance of 1.7 was waaay better. I am about 95% certain the
A/V is the problem but I need to make sure. While it was great to see
the good performance, it is still a big concern to see that A/V might
be having an even bigger impact than it has before. 1.6 also showed
performance improvements but they were less dramatic than some of the
1.7 commands.
Stefan Küng asked me to add svnversion to the benchmark and I have
done so in the latest release.
I would love to see someone do some tests with the WC on local disk vs
network mount (1.6 and 1.7). I tried to do it using some virtual
machines I have access to at CollabNet. The problem is that the
connection of these boxes to the NetApp with our home folders is too
slow. Some of the checkouts (even using 1.6) were running for an hour
and I finally killed the test.
Anyway, overall I am encouraged by the results I am seeing so far. I
look forward to more people running the tests and sharing the results.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
Received on 2011-03-26 19:52:23 CET