On Sun, 2008-08-17 at 22:20 +0200, Neels Janosch Hofmeyr wrote:
> Stefan Sperling wrote:
> > On Sun, Aug 17, 2008 at 02:36:48PM +0200, Neels Janosch Hofmeyr wrote:
> >> Currently, most of these new tests still yield undesirable results. The
> >> course that I aim for is this:
> >>
> >> 1) Commit the new tree-conflicts tests, so that they PASS the current
> >> tree-conflicts branch, even though many test results are still
> >> undesirable. (This patch)
> >
> > Hey Neels,
> >
> > note that there's also XFail(), which can be used to mark
> > tests which are expected to fail.
> >
> > In this case, almost every test would need to be flagged XFail.
> > However, migrating tests from XFail to PASS by changing Subversion's
> > code only might be less confusing than migrating tests from PASS
> > to PASS by changing both Subversion's code and the test in the same go.
> >
> > The net result is the same.
> >
> > But I assume that we don't really know yet what the desired
> > result actually is for many cases, right? :)
> >
> >> 2) One by one, look at the undesirable results and fix them in subsequent
> >> patches. Patches should both fix the situation and make the tests
> >> pass the fixed situation.
> >
> > Maybe make that 3 steps?
> >
> > 1) Have tons of tests that PASS the current state of things (this patch)
> >
> > for test in tests:
> > 2) Figure out what the desired result for $test should be,
> > make $test check for that and mark it XFail.
> > 3) Implement desired behaviour in Subversion's code, making
> > $test PASS, and remove its XFail status.
>
> I would prefer taking the results one by one, adjusting the tests one by
> one, because that clarifies the current development:
>
> - As soon as you have fixed the one issue you were addressing, the tests
> PASS again.
>
> - It is easier to see how one fix affects other situations that hadn't been
> considered. As soon as something "else" is changed, another test fails.
>
> Given the complexity of tree-conflicts, it's nice to have some clarity...
>
> For this way of action, XFail is not useful, since I want the tests to pass
> exactly when I have got the fix right. Also, it would be nice to not have
> segmentation faults around. So that's my first fix candidate.
Did you know that a test marked as "XFail" reports a result of "XFAIL"
when it fails, and "XPASS" ("unexpected pass") when it passes? In this
way we can see when we have fixed a bug for which we wrote a test
earlier.
- Julian
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe_at_subversion.tigris.org
For additional commands, e-mail: dev-help_at_subversion.tigris.org
Received on 2008-08-19 23:51:43 CEST