[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: XFAIL executing make check on SVN 1.6.5

From: Hyrum K. Wright <hyrum_at_hyrumwright.org>
Date: Thu, 29 Oct 2009 16:33:40 +0100

On Oct 29, 2009, at 4:22 PM, Andrey Repin wrote:

> Greetings, Daniel Becroft!
>>> Those are eXpected FAILures; they're supposed to fail if
>>> Subversion is
>>> functioning correctly.
>> My understanding was that XFAIL tests indicated actual test
>> failures that
>> were known, and that should (eventually) be fixed. (Almost like a
>> TODO list)
>> E.g. Scenario X produces result Y, but it should produce result Z.
>> However,
>> making it produce Z is large chunk of work (and potentially a
>> redesign of
>> something).
>> So add a test that runs X and expects Z, and mark it as XFAIL.
>> Eventually,
>> when the work is done to actually produce Z, the test will pass
>> (and no
>> longer be in the XFAIL list).
>> Is this understanding correct?
> No. These are, by word, expected failures. They must fail.
> I.e. if you issue syntactically correct but meaningless or ambiguous
> request,
> Subversion should tell you that you are wrong.

Actually, you are incorrect and Daniel is correct. From http://svn.collab.net/repos/svn/trunk/subversion/tests/README:

   The purpose of XFAIL tests is to confirm that a known bug still
   exists. When you see such a test uneXpectedly PASS, you've probably
   fixed the bug it tests for, even if that wasn't your intention. :-)
   XFAIL is not to be used as a way of testing a deliberately invalid
   operation that is expected to fail when Subversion is working
   correctly, nor as a place-holder for a test that is not yet written.

So tests which test for bogus input, or to make sure that Subversion
produces are correct error, etc. are marked PASS. Tests marked XFAIL
should eventually become PASSing tests.



To unsubscribe from this discussion, e-mail: [users-unsubscribe_at_subversion.tigris.org].
Received on 2009-10-29 16:35:14 CET

This is an archived mail posted to the Subversion Users mailing list.