Karl Fogel <kfogel@newton.ch.collab.net> writes:
> The more general issue is harder to solve. Our test suite takes (on
> my development box) about a half-hour to run over any given RA layer.
> That means that each RA layer we add adds a half-hour of testing time.
> I can run the tests over ra_svn more often, but that will mean
> sometimes *not* running them over dav, or local. More likely, I'll
> run some subset of the tests over each of the ra layers, as a sanity
> check (I should really have started doing that earlier). It's not
> ideal, but an 1.5 hour test run for every change is not sustainable.
> Development would slow down too much.
Like Karl, I don't run the regression tests over all the RA layers for
each change I commit, it does take a long time (although not as long
on Linux as the times I've seen reported for running the tests on
Windows). There are however things that can be done to "optimise" the
procedure. The following applies to a Linux machine as that's what I
use, other OSs probably allow something similar.
I can run all the ra_local, ra_dav and ra_svn tests in half an hour on
this machine, which is three or four years old. To achieve that I use
two techniques. First, I use tmpfs filesystems to make the tests
faster. Second, I use three builds outside the source directory, so I
can run the three tests in parallel. These two techniques complement
one another, the out-of-source build makes using tmpfs easy, and using
tmpfs reduces disk IO allowing parallel testing. If I were to run the
regression tests sequentially, without tmpfs, it would take over two
hours on this machine.
I configure and build using something like
$ svn co http://.../trunk svn
$ mkdir obj
$ cd obj
$ ../svn/configure
$ make
$ make check
About the only difference compared to an in-source build is that when
I want to run a single python test I cannot do
$ cd subversion/tests/clients/cmdline
$ ./basic_tests.py 1
instead I need to specify the path to the python file
$ cd subversion/tests/clients/cmdline
$ /source/dir/svn/subversion/tests/clients/cmdline/basic_tests.py 1
Building outside the source directory does not on its own make the
tests faster -- to do that I use a tmpfs filesystem. This requires a
kernel with tmpfs support, I am using 2.4.20 built with CONFIG_TMPFS=y.
I have lines in my /etc/fstab like
tmpfs /build/dir/subversion/tests tmpfs defaults,users,noauto,exec,size=300m 0 0
where /build/dir is the path to the obj directory, and I use it like
$ mount /build/dir/subversion/tests
$ make mkdir-init
$ make
which mounts the tmpfs filesystem over the tests directory, creates
the tests directory hierarchy, and rebuilds the test stuff. That
doesn't take very long. Now running "make check" it is twice as fast
as before, and the amount of disk activity is much reduced.
Using tmpfs speeds up the tests, but it still doesn't use all the
available CPU. So I can use a second, parallel, build directory
configured to use the same source directory as the first. I build it,
mount a second tmpfs filesystem, and now I can run a second instance
of the regression tests using a different RA layer. I can then go on
to a third directory and run all three RA layers.
This machine does not have 3x300MB of memory so three tmpfs sets of
the regression test output will force it to swap. That's not a big
problem, writing to swap is much less disk intensive than writing to
active BDB databases on disk.
Note, any files created within the tmpfs mount will be lost when it is
unmounted. That's not really a problem since all the files are
generated objects, executables or test output. The test source code
remains on the normal filesystem, so any changes you make are
preserved.
--
Philip Martin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Sun Feb 2 20:59:30 2003