[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Skip-XFail tests accidentally reported as "PASS"

From: Julian Foad <julianfoad_at_btopenworld.com>
Date: Thu, 22 May 2008 17:59:42 +0100

Our test suite can report an XFAIL test as "PASS". This happens when a test is
declared as "Skip(XFail(...".

The bug appears to be that the Skip class fails to delegate its "run_text()"
method to the nested class. I think the attached patch fixes it. The only
immediate effect it has on the basic test suite (ra_local) running on my Linux
system is that it changes:

   PASS: merge_tests.py 85: merge --reintegrate should fail on stale source

to the correct result:

   XFAIL: merge_tests.py 85: merge --reintegrate should fail on stale source

Is there anyone who could review this, please? Preferably somebody who
understands inheritance in Python classes.

FYI, tests that depend on both XFail and Skip/SkipUnless are:
[[[
$ (cd subversion/tests/cmdline/ && grep -i "XFail.*Skip" *.py)
authz_tests.py: XFail(SkipUnless(authz_svnserve_anon_access_read,
authz_tests.py: XFail(Skip(authz_switch_to_directory,
merge_tests.py: XFail(SkipUnless(
                     dont_merge_revs_into_subtree_that_predate_it,
merge_tests.py: XFail(SkipUnless(merge_chokes_on_renamed_subtrees,
special_tests.py: XFail(SkipUnless(diff_symlink_to_dir,
                     svntest.main.is_posix_os)),

$ (cd subversion/tests/cmdline/ && grep -i "Skip.*XFail" *.py)
merge_tests.py: SkipUnless(XFail(reintegrate_fail_on_stale_source),
prop_tests.py: Skip(XFail(revprop_change,
                     svntest.main.is_ra_type_dav),
prop_tests.py: SkipUnless(XFail(invalid_propvalues,
                     svntest.main.is_ra_type_dav),
]]]

That's authz 13,14, merge 93,94, special 10, merge 85, prop 12,26.

I also noticed how Skip and XFail handle the _list_mode_text member in two
different ways - neither of them by overriding the list_mode() method. Is this
why "authz_tests.py list" shows "XFAIL" for 13/14 while the result of running
them is "SKIP", or "merge_tests.py list" shows blank for 85 while the result is
"XFAIL"?

- Julian

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe_at_subversion.tigris.org
For additional commands, e-mail: dev-help_at_subversion.tigris.org

Received on 2008-05-22 19:01:39 CEST

This is an archived mail posted to the Subversion Dev mailing list.