Postmortem: Template unittests are bad & you shouldn't catch Error

[Jacob Carlborg]

On Thursday, 22 October 2020 at 16:05:51 UTC, Mathias LANG wrote:

> You do not test for assert error. Assert errors are logic 
> errors, and if they get triggered, all bets are off.
>
> You disagree with the spec then. The spec says that the stack 
> might not be unwound if an `Error` is thrown.

It's not that simple, you need to read the whole spec ;). The 
spec for unit tests has an exception:

"Individual tests are specified in the unit test using 
AssertExpressions. Unlike AssertExpressions used elsewhere, the 
assert is not assumed to hold, and upon assert failure the 
program is still in a defined state." [1].

Although, it's not clear if that only applies to `assert` 
directly in the `unittest` block or any assert reachable through 
the `unittest` block.

Regardless of what's correct or not, I think there must be a way 
to have unit tests keep running after a failed test. Otherwise D 
will have very poor experience for unit test frameworks. It's 
already bad enough with the default unit test runner that aborts 
the rest of the unit tests inside a module after a failed test 
(it will continue with other modules).

[1] https://dlang.org/spec/unittest.html

--
/Jacob Carlborg