About unittest, test runners and assert libraries

[SealabJaster]

On Monday, 10 May 2021 at 17:20:54 UTC, Zardoz wrote:
> To me, sound logic that any assert library should use the same 
> exception that it's throw by DLang's assert(). This is 
> AssertError. And that tests runners should count any 
> AssertError (or derived) inside a test case as failed, and any 
> other exception as an error. This allows to assert libraries 
> and assert runners work better without needing to be coupled to 
> extend from some internal Exception implemented by a test 
> runner and to have code on assert libraries to handle every 
> test runner on the wild.

I see your point here, but I don't think it's as straightforward.

Imagine a custom array type sort of like this, being tested:

```d
     struct MyArray
     {
         int length;
         string get(int index)
         {
             assert(index < length, "Index out of bounds.");
             return "Hello!";
         }
     }

     unittest
     {
         MyArray array;
         array.length = 2;
         assert(array.get(1) == "Hello!"); // Success
         assert(array.get(2) == "Hello!"); // Assert thrown by 
MyArray.get
     }
```

My point is, would the assert thrown by `MyArray.get` be 
considered a test failure, or a programmer error? The test runner 
wouldn't know since it can't distinguish between the two cases 
with just `AssertError`.

While the Error classes are for unrecoverable errors (programmer 
error/bugs), the Exception classes are for known bad-states that 
can be recovered from. E.g. JSON parsing failure is an Exception, 
not an Error.

So in general, it just feels more natural to catch an Exception 
instead of an Error, since catching an Error is bad practice 
anyway.

If you derive from AssertError, you still need to make your own 
assert functions/throw it manually, like exceptions, so it isn't 
really too much different from just handling exceptions instead.

Just an aside, I use Silly since it's easy to use; unittests can 
still run without it since it only needs a pure-string UDA 
`@("Test name")`, and honestly it makes me wonder why D's builtin 
ability to run unittests is so limited.