Re: HEADS UP: cmdline test factory

Mark Phippard wrote:
> On Sun, Jun 21, 2009 at 2:48 PM, Neels Janosch Hofmeyr<neels_at_elego.de> wrote:
>> This particular test simply checks for successful completion, making sure a
>> certain line of output appears. That's simple.
>>
>> But most tests need to see that all data is in fact in the file system and
>> in the WC metadata exactly as expected.
>
> I do not see how it is possible to do this? Does this depend on the
> test needing to work and then the factory runs the commands, figured
> out what it did and forever records that in Python?
>
> Getting this checking right has always been my biggest PITA when
> writing the JUnit JavaHL tests which are more or less the same as the
> Python tests.
>

Hi Mark,

there's not much mystery in the cmdline factory. It takes svn commands, runs
them one by one and each time takes stock of the WC metadata, disk and
status output (in the same way that the cmdline tests currently would
process run_and_verify_* calls). All that it does is write this information
down in cmdline test py.

It will simply write down what it finds.
If svn fails, factory will write *that* down and make the test expect a
failure. (The resulting test script passes if it finds the same failure)

Now, if you are writing a test for some future feature, you can still run
factory as a first step. After that you need to tweak the resulting test
lingo to reflect the actually desirable behaviour.

But, hopefully, you saved a lot of time by having the main thing written
down already. That's all. :)

(Let me repeat for everyone that factory writes py, as a first step. You
still need to put the py where it belongs, otherwise the whole test is
useless. Factory runs will never fail, they will adapt to whatever svn spits
out. No-one should ever commit a test that still has factory calls in them.
Factory calls are temporary, just a handful of helping pyxies.)

~Neels

------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=2363997