Re: subversion client test suite

Mo DeJong writes:
> On 7 Mar 2001, Ben Collins-Sussman wrote:
>
> > Greg Hudson <ghudson@MIT.EDU> writes:
> >
> > > As a site integrator, I find it obnoxious when packages rely on perl
> > > for any part of the build or regression test procedure. (Regression
> > > tests are valuble to site integrators as well as developers.)
> >
> > I don't understand your objection. Why is it obnoxious?
>
> Software should strive to depend as little as possible
> on "external" bits. That just makes it harder to get
> up an running.

I generally agree with this sentiment. However, I think it is safe to
say that Perl is on the vast majority of systems. Sadly, Python is
not as popular (yet!). Even for systems that Perl is not on, I think it is
safe to say that Perl is one of the first things to go on those
systems. I could very well be smoking my body weight in crack, but I
like to think that Perl is a standard tool these days and can be
considered about as "external" as vi and ksh or bash.

> > I mean, seriously, are there *any* systems out there that don't come
> > with perl pre-installed these days? Or, are there any site
> > integrators out there that don't use perl in *some* respect to do
> > their jobs (thereby guaranteeing that perl is available)?
>
> Many systems come with Perl, many do not. The problem shows
> up when you depend on having that piece of software
> installed before yours will work. This is even more
> critical when it comes to your tests cases. Suppose
> you run the test cases and they all fail. What is
> at fault? Is the version of Perl (or whatever,
> I am not attacking Perl here) installed on the system
> broken? Is Perl installed at all? Is it a different
> version than your test cases expected? Is it your code?

Don't look now, but you just provided an argument against using Tcl.
Just s/Perl/Tcl/g in the above paragraph. :)

> > It seems like you're worrying about extreme edge-cases.
>
> I don't think they are edge cases. If you want to
> depend on Perl then it should be included in the
> source tree. If folks have that exact same
> version installed on the system, they can
> pass --with-perl=PATH to use it. Otherwise, a
> known version should be built from the source
> tree. You don't have to install it, but you
> should use it to run the test cases.

I think this itself is a bit extreme. If we should decide to write
the test suite in Bash, does that mean we need to include Bash in the
source tree? Is Bash standard enough that we don't have to worry
about version compatibility? Rhetorically speaking, the same could be
asked of Python or Tcl (Jacks). I mean, I know for a fact that the
Perl and Python developers go to a lot of trouble to insure backward
compatibility. I cannot speak for Tcl, but I would assume the same.

So really, no matter what interpreted language is used (if any), it
should be written in a safe and generic way such that the version does
not matter. Outside of the major differences betwean Perl 4 and Perl
5, for example, it is a small thing to write clean, maintainable,
backward compatible code (for those who know). Again, I would assume
the same for Python or Tcl.

Maybe I am missing something, but saying that we should include Perl
(Python or Tcl) in our source tree is overkill, mainly because it is a
common tool. It's like saying we should include an ANSI C compiler in
the Subversion source. If our choice was an uncommon tool (say
Jacks), then I would agree that it should be included. I guess it
depends on a consensus of what is considered standard and, even then,
acceptable.

> > Writing code in sh or C to examine the contents of SVN/ files runs the
> > risk of being extremely awkward and painful for all current and future
> > svn developers; whereas perl is the perfect tool for such a task.
>
> Writing regression tests in C is a bad idea, no argument there.
>
> > The real issue is the trade-off here. Even if I grant you that an
> > perl-based test suite might annoy some small group of site
> > integrators, isn't this a far lesser evil than annoying all the svn
> > developers? (Remember that the regression tests will -most- often be
> > run (and written) by developers, which is a larger group IMO.)
>
> The test suite is for developers, but it needs to be easy
> to use so that people can verify that a build or new
> port is working as expected.

So what is easiest for "people"? Perl, Python, Tcl or something else?

I mean, if we are talking about Joe User running the tests, then all
they have to do is run "make check" or whatever the docs say,
(provided they have the requisite language available). :)

If we are talking about Joe Developer writing tests, then it might be
helpful to consider how pervasive each tool is in the community. I
think "ease of use" certainly correlates to "familiarity with" in this
case. That is, I am willing to bet that developers' familiarity with
the three mentioned languages rates like so (greater familiarity to
lesser):

Perl, Python, Tcl

or maybe

Perl, Tcl, Python

> Mo DeJong
> Red Hat Inc

So it looks like we have to think about these things before actually
deciding on the language:

* Is an "external" language necessary and acceptable for this task?

* If so, is it reasonable to expect people to already have or install
  the required language?

* If the second point is not reasonable, is it reasonable (or
  necessary) to include that language in Subversion?

Once the above are answered:

* What language (if we go there) best serves the Subversion community
  (users and developers) with regard to "ease of use"?

-- 
Lee P. W. Burgess  <<!>>  Manipulate eternity. Power is a symphony:
Programmer         <<!>>  elaborate, enormous, essential.
Red Bean Software  <<!>>  Dream the moment with a fiddle in summer 
lefty@red-bean.com <<!>>  and a knife in winter.