[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

When to commit failing tests (was: Re: svn commit: r17032 - trunk/subversion/tests/clients/cmdline)

From: Peter N. Lundblad <peter_at_famlundblad.se>
Date: 2005-10-26 20:24:45 CEST

Hi,

On Wed, 26 Oct 2005, Erik Huelsmann wrote:

> > On 10/26/05, Peter N. Lundblad <peter@famlundblad.se> wrote:
> > On Wed, 26 Oct 2005, Erik Huelsmann wrote:
> >
> > >> On 10/26/05, lundblad@tigris.org <lundblad@tigris.org> wrote:
> > >> Log:
> > >> * subversion/tests/client/cmdline/revert_tests.py
> > >> (test_list): Make revert_from_wc_root XFail.
> >
> > > Actually, no, that's not the purpose of XFail, as far as I conceive
> it:
>
> > > Tests should be marked XFail if a feature currently isn't
> implemented
> > > or behaving as it should, but that fact is recognised and agreed
upon
> > > [ie postponing a fix is acceptable].
>
> > This has come up before. Yes, it is a regression that needs to be
> fixed.
> > But what's the point in making all nightly builds fail and all others
> > tests fail? Either you just fix it, or add an issue as a 1.4.0
> blocker.

> Well, the point being that if there's a regression, normally, failing
> tests indicate it. In this case, there hasn't been the same
> indication. Other devs would have missed it if they weren't on irc
> yesterday. Also, I think there's a difference between a once correctly
> implemented behaviour not working anymore and a newly implemented
> feature not working entirely as expected.

You normally don't commit something that you know have failing tests. If
the nightly build fail, because you did something platform-dependent or
something, then that's another story. I really don't think knowingly
commit failing tests non-XFail is a good way of alerting people that there
is a regression. You can post to dev@, file an issue or just fix it in a
few days.

The problem with your approach is that it slows down things for everyone
else. When I want to run tests and get a failure, I have to examine and
dscover that it was just that other regression that still fails. If I made
a commit, I have to examine the nightly failure mails, because I can't
just assume that it is your test that fail. I think that's a waste of
time.

Regards,
//Peter

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Wed Oct 26 20:26:24 2005

This is an archived mail posted to the Subversion Dev mailing list.