Test case confusion.

Hi all.

I have started to poke around in the test suite again and I am running into
some things that just don't make sense to me. I am hoping the person
that designed these tests can shed some light on a couple of things.

For one thing, why is there this sandbox() method in each .py test file?

def sandbox(x):
  return "basic_tests-" + `test_list.index(x)`

A call like:

sbox = sandbox(basic_status)

Will return a string like "basic_tests-0" which will be used to create a unique name
for the repo and checkout dirs for the test case. Why would we want to do that instead
of just assuming that each test function has a globally unique name and using the test name
as the dir name?

The current implementation makes it hard to map from the directory names in
subversion/tests/clients/cmdline/repositories to the actual test cases. You need
to manually count the number of entries in the test_list to find the right one.

Instead of:

  sbox = sandbox(basic_commit)

We could just use:

  sbox = "basic_status"

That would make it really easy to figure out which test a given repo and
WC belonged to.

The other problem I am having has to do with running a single test from the
make check command line. The make check rule start off like so:

check: $(TEST_DEPS) @FS_TEST_DEPS@
        @logfile=`pwd`/tests.log ; \
        echo > $$logfile ; \
        failed=no ; \
        list='$(TEST_PROGRAMS) @FS_TEST_PROGRAMS@'; for prog in $$list; do \

So, I can run a single test case from the command line by doing this:

make check TEST_PROGRAMS="" \
FS_TEST_PROGRAMS="subversion/tests/clients/cmdline/basic_tests.py"

As you can see, that is quite a bother. What if we made a small change
to the Makefile so that is easier?

% make check TESTS="subversion/tests/clients/cmdline/basic_tests.py"

It could be implemented something like this (not a real patch)

Index: Makefile.in
-----------------------
+# Users can run 'make check TESTS=...' from the command line to run
+# a single test or set of tests
+TESTS = $(TEST_PROGRAMS) @FS_TEST_PROGRAMS@
+
 check: $(TEST_DEPS) @FS_TEST_DEPS@
        @logfile=`pwd`/tests.log ; \
        echo > $$logfile ; \
        failed=no ; \
- list='$(TEST_PROGRAMS) @FS_TEST_PROGRAMS@'; for prog in $$list; do \
+ list='$(TESTS)'; for prog in $$list; do \

cheers
Mo

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Sat Oct 21 14:36:51 2006