So I spent some time Tuesday talking to Ben and Karl about the client
test suite. Basically what is need is something that is fully
automated rather than partially automated.
What we have is two shell scripts that invoke the client for various
operations (checkout, checkin, update, etc.). This is satisfactory in
that the return value for each invokation is checked. That is, as
long as each client operation does not fail, the test passes. This is
also somewhat useful because some tests are dependent on the success
of previous tests.
The test suite is not satisfactory in at least two ways:
* The real, qualitative result of each client operation is not
checked; only the return value of the client process is checked and
we take for granted that things are really working like we expect
them to. If I do a checkout, a checkin and then update, I want to
know positively that the checkin generated a delta with the correct
information; likewise, the update should be verified to have
correctly updated my "working copy".
* Each test should have the exact same result whether it is run alone
or as part of the suite; tests should not be dependent on other
tests. Like, I should not have to run test 1 and test 3 to get the
necessary state for test 5 to succeed. Tests 1, 3 and 5 should
really call the same functions from a library.
Now, I have volunteered to take on the task of cleaning up the client
test suite. As mentioned, the tests are run by two shell scripts.
Note that the client test suite does not utilize the test framework
used by the rest of Subversion. Since the client is a separate
program, it needs to be tested on a per-invocation basis. And the
real testing is actually looking at the files affected by that
invocation to make sure they were modified in the correct manner.
I think this could be done in Bash; but might be more trouble than
it's worth (harder to maintain and extend?). This could also be done
in C, but why bother? I think there are better tools than C for
file/string testing, grepping and manipulation, especially for this
particular task. You all see where I am going with this.
The way I see it, I have two choices: Perl or Python. I am more
fluent in Perl, but I like Python more. I would just as soon use
Python, but I wanted to put it to the list before acting. So I am
looking for *constructive* feedback regarding what other people think
is the Right Tool For This Job.
By "constructive feedback", I mean I am asking for concise, objective
statements about how the use of Perl or Python will better serve
Subversion. I am NOT looking for Perl or Python advocacy. I already
know both languages to the extent that I know what each does and what
each does better or worse than the other. Anything any of you can say
about either language, we have all heard already. So please, save the
evangelization for some other time and place.
A couple of issues to think about are portability and maintenance.
Portability is probably paramount. A close second, is the combination
of clarity, maintainability and the ability to easily extend what will
be written. Most of that is basically good programming practice and
largely independent of language. This new test suite is very
important, so let's not make typical assumptions about one language or
Do not think of this as some sort of competition between Perl or
Python. What we want is what wins for Subverison. If you think that
is Bash or C or something else, then say so and back it up.
Lee P. W. Burgess <<!>> Manipulate eternity. Power is a symphony:
Programmer <<!>> elaborate, enormous, essential.
Red Bean Software <<!>> Dream the moment with a fiddle in summer
firstname.lastname@example.org <<!>> and a knife in winter.
Received on Sat Oct 21 14:36:25 2006