Extending unittests [proposal] [Proof Of Concept]

[Johannes Pfau]

Am Fri, 21 Sep 2012 23:15:33 +0200
schrieb Jens Mueller <jens.k.mueller at gmx.de>:

> > I like the BOOST unit test library's approach, which has two types
> > of "assert": BOOST_CHECK and BOOST_REQUIRE. After a BOOST_CHECK
> > fails, the test keeps running, but BOOST_REQUIRE throws an exception
> > to stop the test. When testing a series of inputs in a loop, it is
> > useful (for debugging) to see the complete set of which ones succeed
> > and which ones fail. For this feature (continuation) to be really
> > useful though, it needs to be able to output context information on
> > failure (e.g. "during iteration 13 of input group B").
> 
> This leads us to the distinction of exceptions and errors. It is safe
> to catch exceptions but less so for errors. At least it is far more
> dangerous and less advised to continue execution but should not be
> prohibited I think.
> 
> Jens

I don't know, I'd expect Exceptions, Errors and assert to always exit
the current scope, so I'd expect this to always work:

unittest
{
    throw new Exception();
    assert(1 == 2);
    auto a = *cast(int*)null; //should never be executed
}

I think a proper unit test framework should supply an additional check
method:
unittest
{
    check(a == 1); //always continue execution
    check(b == 2); //always continue execution
    check(c == 3); //always continue execution
}

This can be implemented in library code: Have a module level array of
failed checks, then let check append to that array. Analyze the result
after every unit test run and empty the array. Then call the next test
and so on.