Re: Status report of testsuite on trunk @ r24830

On 4/29/07, Lieven Govaerts <lgo@mobsol.be> wrote:
>
> I wrote a small script tonight to compare the status of the test suite
> on trunk and the 1.4.x branch. Main goal is to check, before the release
> of 1.5, that we're not introducing known regressions and fix all the
> tests we possibly can.
>
> Attached is the output of the script of trunk & branches/1.4.x at r24830.
>
> Short summary for the impatient:
> 1. trunk has 795 tests compared to the 662 tests of 1.4.x (133 new tests!)
> 2. There are 3 regressions introduced on trunk, which as far as I'm
> concerned are blocking issues for the next release.
> 3. There are 19 new non-passing tests added on trunk, we might want to
> check which of those need fixing/implementation.
>
> Lieven
>
> Nr. of tests on 1.4.x: 662
> Nr. of tests on trunk: 795 (= +133)

Lieven,

What would you think about adding some small stats output to the
existing suite? Maybe change the output to something like this:

Running all tests in compat-test [1/50]...success [14/0/1/0/2]

Where the numbers are [PASS/FAIL/SKIPPED/XPASS/XFAIL]

We are trying to get our QA team at CollabNet to use the test suite
and they are wanting information like this from the output. It does
not necessarily matter how it comes out or is formatted, the above was
just a suggestion.

What do you think of this idea?

What would be the best way to do it?

-- 
Thanks
Mark Phippard
http://markphip.blogspot.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org