[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: copy_test, update crash and dirent_uri failure on Windows

From: Julian Foad <julian.foad_at_wandisco.com>
Date: Thu, 07 Jan 2010 12:28:50 +0000

On Thu, 2010-01-07 at 10:20 +0800, Ed wrote:
> Julian Foad wrote:
> > The report of "success" indicates a bug in the test harness. We had (and
> > still have) similar problems with the Linux test harness.
> I'm still having slight difficulties understanding these tests
> in the regards that an XFAIL is a success despite it being
> expected to fail.

See the description of XFAIL in the section "How to write new C tests"
<http://svn.apache.org/repos/asf/subversion/trunk/subversion/tests/README>. XFAIL means "We know this test fails, and are happy to live with that for some longish time." At the level of running a whole suite of tests, we want to consider the result as successful if the only failures were expected failures, so that's what the test harness does.

> But for some reasons, Linux tests don't report svn segfaulting,

When you say they "don't report it", what they should do is catch svn
segfaulting and treat that as a test failing, the same as if svn
produced any kind of unexpected results. If the test in which
svnsegfaults is marked "XFAIL", then it should report the result as
"XFAIL" and therefore effectively a pass.

Sometimes a test marks XFAIL fails for a reason other than the expected
reason - e.g. it segfaults whereas the original expected failure was
some particular unwanted commandline output. We don't have a way of
detecting this "fails for the wrong reason" scenario.

> unlike Windows tests which svn.exe crashes. Guess I should
> mark that as an operating system difference? Anyway, thanks
> for the clarification.

- Julian
Received on 2010-01-07 13:29:28 CET

This is an archived mail posted to the Subversion Dev mailing list.