Re: expected failures shouldn't raise alarms

On Wednesday, August 21, 2002, at 09:28 PM, Karl Fogel wrote:

> Brane, thanks much for the XFAIL stuff. I've just one issue with it:
> when "make check" is printing out the summary results, it treats
> expected failures as "FAILURE"s. This is needlessly alarming -- the
> whole *point* of the all-caps word "FAILURE" is to stand out and let
> the programmer know that something's wrong, so don't commit now.
>
> Tests that generate expected failures should print "success". They
> behaved as expected, so they succeded.

Yes, quite so. But this does raise another question. What happens when a
test that is expected to fail doesn't?

One way to handle this might be to print the words success or failure to
indicate whether or not the test passed, and to use all caps to indicate
that this was unexpected:

success: test passed, all is well.
failure: test failed, as expected.
SUCCESS: test passed, huh?
FAILURE: test failed, you broke something.

Cheers,

Colin

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Thu Aug 22 07:38:47 2002