Extending unittests [proposal] [Proof Of Concept]

[Jens Mueller]

Johannes Pfau wrote:
> Am Fri, 21 Sep 2012 23:15:33 +0200
> schrieb Jens Mueller <jens.k.mueller at gmx.de>:
> 
> > > I like the BOOST unit test library's approach, which has two types
> > > of "assert": BOOST_CHECK and BOOST_REQUIRE. After a BOOST_CHECK
> > > fails, the test keeps running, but BOOST_REQUIRE throws an exception
> > > to stop the test. When testing a series of inputs in a loop, it is
> > > useful (for debugging) to see the complete set of which ones succeed
> > > and which ones fail. For this feature (continuation) to be really
> > > useful though, it needs to be able to output context information on
> > > failure (e.g. "during iteration 13 of input group B").
> > 
> > This leads us to the distinction of exceptions and errors. It is safe
> > to catch exceptions but less so for errors. At least it is far more
> > dangerous and less advised to continue execution but should not be
> > prohibited I think.
> > 
> > Jens
> 
> I don't know, I'd expect Exceptions, Errors and assert to always exit
> the current scope, so I'd expect this to always work:
> 
> unittest
> {
>     throw new Exception();
>     assert(1 == 2);
>     auto a = *cast(int*)null; //should never be executed
> }

That should definitely be the default.
I think dmd also warns about such code.

> I think a proper unit test framework should supply an additional check
> method:
> unittest
> {
>     check(a == 1); //always continue execution
>     check(b == 2); //always continue execution
>     check(c == 3); //always continue execution
> }
> 
> This can be implemented in library code: Have a module level array of
> failed checks, then let check append to that array. Analyze the result
> after every unit test run and empty the array. Then call the next test
> and so on.

gtest also has ASSERT_* and EXPECT_*. I used EXPECT_* rarely.
I think it complicates matters for little reason. Because in D you then
have assert, enforce, and check. I would prefer if assert and enforce
are the only ones. Though semantics for enforce and check are a little
different. An exception is like a check. It may fail and can be handled.

Jens