> Hi Mo... I haven't looked closely at this package yet. It looks like
> you've written a system that is an alternative to our 'make check'
> rule -- it walks through the source tree, runs any test scripts it can
> find, logs them, and looks for regression problems. Is that right?
Well, yes. Except that it does a whole lot more.
> My question is: if this is indeed what you've written, is the system
> able to run all of our C-library API tests,
The tests implemented in C that I looked at would execute some
C code and then exit the process with a 0 exit status if
the test passed. That sort of test would be easy to add,
but my long term goal is to replace all these subprocess
tests with in-process tests. One should be able to write
the tests in a scripting language, not in C, and that
nifty swig wrapper tool is the the way to do that.
> as well as our external python tests?
The stuff in clients/cmdline that I looked seemed
to be similar to the exec based C test framework.
My goal for svntest was to replace the C framework,
so it seems these two new frameworks would be
in competition. There is nothing wrong with a
little competition, I just don't want to see
multiple frameworks in actual use, that
would just confuse the heck out of anyone
that wants to add a test.
> If so, that's great... all of our existing tests
> programs have the same user interface anyway. It shouldn't be hard to
> do.
>
> The reason I ask is that your sample 'help subcommand' tests are
> written in tcl like the rest of your system. Looking quickly over
> them, I wasn't able to tell if your system is language-unbiased about
> the tests it runs.
Well, if you are talking about just execing a subprocess and
checking the exit code, that is about as "language-unbiased"
as it gets. Of course, it is also not very interesting.
If you ask me, in-process testing is the way to go.
That would provide a nice test of the swig Tcl binding
layer as well as make the tests much easier to write.
Does that mean people might have to learn some Tcl,
well sure. One would have to learn Python to use the
python based test layer. If folks are really keen on
writing tests in Python, I guess you could always
re-implement the test framework I posted in Python.
It seems like a waste of time to me, but everyone
has different tastes.
At the end of the day, the ease with which someone can
look at the tests and figure out what they are doing
is the thing that matters.
Mo DeJong
Red Hat Inc
Received on Sat Oct 21 14:36:28 2006