std.experimental.testing formal review

[Dicebot via Digitalmars-d]

On Saturday, 12 September 2015 at 14:50:32 UTC, Jacob Carlborg 
wrote:
> On 2015-09-12 15:34, Dicebot wrote:
>
>> I also don't like mixing unittest and higher level functional 
>> tests
>> (with setup and cleanup phases) into the same buckets - this 
>> doesn't fit
>> nice with D module system. Latter should be placed in a 
>> separate
>> modules/package to avoid being picked up by rdmd & Co when 
>> compiled as
>> dependency.
>
> Not sure I understand the problem. Does this prevent one from 
> writing functional tests in a completely separate directory?

On related topic - there are 2 things that are currently missing 
in `TestCase` when applied for functional test purpose:

1) being able to mark test case as fatal (i.e. if internal 
handshake or sanity check fails there is no point in trying to 
run other tests)
2) being able to do weak ordering of tests (by defining strict 
sequence of groups so that parallelization/randomization only 
happens within such group) - I have used something as simple as 
numerical priority value so far for my needs