On Wed, Feb 02, 2011 at 04:12:07PM +0100, OBones wrote:
> Neil Bird wrote:
> > I couldn't use the version from trunk/1.7 as it differs too
> >much. I will try to submit the patch for someone's perusal at
> >some point, but I couldn't properly test is as for some reason my
> >build of svn out of 1.6.x svn (even before modifying it) fails
> >'make check': all the tests pass, but I get a load of XFAIL lines
> >I don't understand.
> >
> > AFAICT, the XFAILs I get after my patch are the same as before,
> >but it's not the warm PASS feeling I was hoping for.
> XFAIL stands for "eXpected to FAIL", meaning that if SVN is doing
> good, then the test should fail.
> For instance, you expect svn not to delete an existing file on
> checkout, so you write your test expecting it to fail the checkout.
> If it succeeds, the test has not failed as expected.
That's not quite right. Behaviour like that would be verified
via a PASS test. The test would FAIL if svn overwrote the file.
XFails are used to flag known bugs or undesirable behaviour which
cannot be fixed at present (e.g. there are a couple of XFAIL tests
for tree conflict handling -- we'd like to do better, but can't at the
moment). Once the bug is fixed, the test will XPASS (unexpected PASS),
and we switch it to PASS then.
Usually, there is an open issue corresponding to an XFail test.
A test may also be marked XFAIL temporarily to keep the buildbots
quiet about a test that is expected to fail for a while during
on-going development. Marking a test XFAIL is easier than changing
the test expectations and changing them again later. It's a one-line
change vs. potentially larger changes.
Ideally, there wouldn't be any XFAILing tests. Most XFAIL tests indicate
that a developer had to punt on fixing a problem, or postpone resolution
of a problem because of other blocking issues.
Stefan
Received on 2011-02-02 16:40:47 CET