std.experimental.testing formal review

[Atila Neves via Digitalmars-d]

On Sunday, 13 September 2015 at 09:59:18 UTC, Dicebot wrote:
> On Saturday, 12 September 2015 at 14:50:32 UTC, Jacob Carlborg 
> wrote:
>> On 2015-09-12 15:34, Dicebot wrote:
>>
>>> I also don't like mixing unittest and higher level functional 
>>> tests
>>> (with setup and cleanup phases) into the same buckets - this 
>>> doesn't fit
>>> nice with D module system. Latter should be placed in a 
>>> separate
>>> modules/package to avoid being picked up by rdmd & Co when 
>>> compiled as
>>> dependency.
>>
>> Not sure I understand the problem. Does this prevent one from 
>> writing functional tests in a completely separate directory?
>
> On related topic - there are 2 things that are currently 
> missing in `TestCase` when applied for functional test purpose:
>
> 1) being able to mark test case as fatal (i.e. if internal 
> handshake or sanity check fails there is no point in trying to 
> run other tests)

I'll see if I can add this before the end of the review phase.

> 2) being able to do weak ordering of tests (by defining strict 
> sequence of groups so that parallelization/randomization only 
> happens within such group) - I have used something as simple as 
> numerical priority value so far for my needs

How about I change the name from @singleThreaded to @serial and 
make sure it works with randomised runs as well (it probably 
already does, I just have to check)?

Atila