Testing in the D Standard Library

[Sebastien Alaiwan via Digitalmars-d-announce]

On Monday, 23 January 2017 at 01:52:29 UTC, Chris Wright wrote:
> On Sun, 22 Jan 2017 20:18:11 +0000, Mark wrote:
>> Have you considered adding randomized tests to Phobos?
>
> Randomized testing is an interesting strategy to use alongside 
> deterministic testing. It produces more coverage over time. 
> However, any given test run only has a fraction of the coverage 
> that you see over a large number of runs.
>
> In other words, if the randomized tests catch something, you 
> don't know who dun it. This is generally considered a bad thing.
>
> Phobos does have some tests that use a PRNG. They all use a 
> fixed seed. This is a shortcut to coming up with arbitrary test 
> data; it's not a way to increase overall coverage.
>
> I think the right way to do it is to have a nightly randomized 
> test build, but since I'm not willing to do the work, I don't 
> have much say.

This. So much this.
Unit tests that loop over many randomly generated input test 
vectors are just a waste of everybody's CPU time.

Don't get me wrong: fuzzing is also necessary. But it relies on 
an arbitrary time limit, which is hardly compatible with keeping 
a test suite fast. Which means it should be done in a another 
validation process than unit tests.