Re: Test XFail/Skip policy

On Tue, Mar 10, 2009 at 17:17, Branko Cibej <brane_at_xbc.nu> wrote:
> Bert Huijben wrote:
>> If the build fails here at work, it must be fixed ASAP.. not half a week later.
>> (These patch tests started failing last Saturday; the moment the branch was merged to trunk)
>>
>
> I don't see a reason why work should be any different than this project.

Because this is a VOLUNTEER project.

People don't have a ton of time to work on this stuff. They don't have
a paycheck on the line.

> So I'm not happy that things have been failing for so long.
>
>>>  We're not reinventing the wheel WRT test results here; we inherited the
>>> PASS/FAIL/XPASS/XFAIL from older systems that have a long history of
>>> using them.
>>>
>>
>> This all assumes somebody investigates problems until they are a stable PASS or a stable FAIL, which is not the case here.
>>
>> I think Arfrever can use some help to get these tests in either an Pass or an XFail. Until that point is reached the test code doesn't belong on trunk. At least not in a way that it can fail the buildbots.
>>
>
> I agree -- like Greg said, revert it or fix it. :)

Reverting it would be dumb. Fixing it is: skip the test.

When somebody gets some time to work on the tests, *then* it can be
moved to something better.

I forget who suggested it, but an alias for Skip() would be nice:

  NeedsWork(...)

It could be reported separately, too, and we make sure it hits zero
before release. But it doesn't cause buildbots to light up.

Cheers,
-g

------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=1303860
Received on 2009-03-10 17:22:43 CET