[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: [#4840] Merge assertion failure in svn_sort__array_insert

From: Julian Foad <julianfoad_at_apache.org>
Date: Wed, 8 Jan 2020 10:14:42 +0000

Daniel Shahaf wrote:
> Julian Foad wrote on Tue, Jan 07, 2020 at 09:47:48 +0000:
>> For now, I propose to make each test use a repeatable sequence, independent
>> of the other tests. I think that will be enough; the options I mentioned
>> can be added if and when there is a demand for them.
>
> I don't understand why you're fixing the seed.
>
> For repeatability, all you need is to output the seed to stdout and to be able
> to re-run the tests with a particular value of the seed:

The suggestion was that we should prefer the regression test suite to be
deterministic, running the same fixed set of tests each time. Repeating
by default, not just repeatable on request.

The other way (non-repeating by default, and repeatable on request) is a
possible alternative that we could choose. Indeed, a few (six or so)
other tests in the suite are currently written that way. However we
don't really have the test infrastructure for handling that nicely. We
could add some, starting with the pseudocode you posted, but we haven't
yet and it would probably want a bit more than that to make an overall
good experience.

> Conversely, the effect of fixing the seed is to make every single test run test
> exactly the same sequence of API calls on the same sequence of inputs. That
> means the test is not merely repeatable, but also deterministic

Yes, like nearly all of our test suite.

> — which begs
> the question, why do we need a PRNG at all? What do we gain

We gain the ability to test a large, diverse set of combinations of
inputs. Call it an "test case generator" instead of a "PRNG", if that
helps.

> in exchange for the
> costs and risks associated with using a PRNG in the test suite?

What?

- Julian
Received on 2020-01-08 11:14:44 CET

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.