Branko Čibej wrote:
> Lieven Govaerts wrote:
>> Branko Čibej wrote:
>>> Can someone please remind me why those two serf tests aren't XFAIL?
>> Why would we do that??
>>
>> Those two tests you are referring too are fixed, but now 10 other
>> tests are failing. Those are fixed on trunk too and a backport branch
>> is ready, but too late for being included in 1.5.1.
>>
>> If people are not interested in seeing the test results of ra_serf,
>> why even bother running those tests while signing?
>
> I don't see the connection here? XFAIL was designed to flag tests that
> are known to fail in certain configurations. It's valid to have XFAIL
> on one branch but not on another.
I agree, but what do you hope to achieve with marking these 10 failing
tests as XFail?
XFail is a way to silence test errors that we are not planning to solve
in the near future. There are only two situations that I can think of
for marking a test as XFail:
1. The test is marking the 'acceptance criteria' for a new feature in
development. XFail here means: svn should behave like this but we didn't
implement the necessary functionality yet. This could be module
specific. Suppose we had a test for PKCS#11 support, than that should be
marked as XFail over ra_serf as this feature was only implemented in
ra_neon.
2. Someone reported an issue and a test is added to reproduce it and
indicate expected behavior. XFail here means that the issue is known,
but unfixed.
These failing tests do not fall into one these two categories, because
all of the tested functionality used to work in trunk and 1.5.0, meaning
we are introducing regressions in svn 1.5.1 over ra_serf.
Do you agree that we should never mark tests that fail because of
regressions as XFail? Even if it's in an experimental module as ra_serf?
The only effect of marking these tests as XFail, is that we are hiding
these regressions for our users and ourselves indefinitely. It's not
that we review the list of XFailing tests every week or so.
Lieven
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe_at_subversion.tigris.org
For additional commands, e-mail: dev-help_at_subversion.tigris.org
Received on 2008-07-25 18:15:05 CEST