Unit tests, asserts and contradictions in the spec

[Paul Backus]

On Wednesday, 6 February 2019 at 21:40:38 UTC, H. S. Teoh wrote:
> [...]
>
> In unittests, however, we've had the convention of using 
> asserts for test failures since the early days.  Generally, UB 
> caused by asserts in this context probably wouldn't have the 
> same risk as above, but it does bring into question the wisdom 
> of continuing to run after a unittest has failed.  The trouble 
> is that the assert may have happened deep inside the call 
> stack, and since it throws AssertError, that means stack 
> unwinding code is potentially never run [...]
>
> [...]
>
> But this means that if the contract failed because of some 
> deeply-nested assert while evaluating the contract. E.g., the 
> contract calls some function which in turns calls a bunch of 
> stuff, and somewhere in there an assert fails. AssertError is 
> thrown [...]

It seems like the fundamental problem here is the failure 
distinguish between a failed assertion *in the actual body* of a 
contract or unit test, and one that occurs deeper in the stack, 
in code called by the contract or unit test. Only assertions that 
are part of the unit test/contract body itself should have their 
failure treated specially, since they're the only ones written 
with the special unit test/contract assertion semantics in mind.

Probably the easiest way to distinguish different types of 
assertions would be to have them throw different AssertError 
subclasses (e.g., UnittestAssertError, ContractAssertError), 
which the runtime could then check for specifically. This would 
allow existing code that expects AssertError, like custom test 
runners, to continue to "work", with the aforementioned caveats.