On 21.02.2011 13:12, Philip Martin wrote:
> Branko Čibej <brane_at_e-reka.si> writes:
>
>> We should not be optimising tests for performance over clarity, ever. In
>> other words -- don't combine them. Anyone who has trouble with tests
>> taking too long can use a RAMdisk, --bdb-txn-nosync, and other tweaks to
>> make the tests run faster.
> I do that. The time to run the testsuite keeps growing.
>
>> I think you've all gone way off track here. Reality check, please? What
>> is the purpose of tests? To validate the behaviour of the code, or to
>> make developers' life easier?
> To make it easy for developers to validate the code :)
>
> Like all code it requires a judgement about how it should be organised.
> I agree that complicated tests should be separate, but I think checking
> that:
>
> "svn revert foo"
>
> and
>
> "svn resoved foo"
>
> both report "foo is not a local path" doesn't require two completely
> separate tests.
>
> How far do you think we should go? A dozen separate tests for revert
> alone:
>
> "svn revert versioned_and_modified"
> "svn revert versioned_and_unmodified"
> "svn revert url"
> "svn revert unversioned"
> "svn revert versioned_and_modified versioned_and_unmodified"
> "svn revert versioned_and_unmodified unversioned"
> etc
It's certainly easier to read the test output that way. :)
You could argue that tiny tests that do not modify a sandbox can be
merged together. Of course, if there's a bug in the implementation and
one of those subtests /does/ modify the sandbox, causing another down
the line to fail for no obvious reason, that hardly makes life easier
for the poor sod who has to analyse that mess.
There's a balancing act here. Talk about a "global sandbox" reused by
several test cases definitely makes my hair stand on end and I do
recommend against that. Combining several small tests into a single
testcase is a different and slightly more palatable approach, as long as
everyone agrees to be the poor sod that has to analyse the fallout. :)
-- Brane
Received on 2011-02-21 13:19:31 CET