Re: [#4840] Merge assertion failure in svn_sort__array_insert

Julian Foad wrote on Tue, Jan 07, 2020 at 09:47:48 +0000:
> For now, I propose to make each test use a repeatable sequence, independent
> of the other tests. I think that will be enough; the options I mentioned
> can be added if and when there is a demand for them.
>
> How does that sound?

I don't understand why you're fixing the seed.

For repeatability, all you need is to output the seed to stdout and to be able
to re-run the tests with a particular value of the seed:

    if "mergeinfo-test.c seed" in os.environ:
        seed = int(os.getenv("mergeinfo-test.c seed"))
    else:
        seed = random.randint(0, 2**32 - 1)
    print("seed =", seed)
    run_test()

(Yes, I know we're talking about C tests; it's pseudocode.)

Conversely, the effect of fixing the seed is to make every single test run test
exactly the same sequence of API calls on the same sequence of inputs. That
means the test is not merely repeatable, but also deterministic — which begs
the question, why do we need a PRNG at all? What do we gain in exchange for the
costs and risks associated with using a PRNG in the test suite?

Cheers,

Daniel
Received on 2020-01-07 16:59:34 CET