Unit Testing in Action

[John Carter]

On Friday, 20 October 2017 at 14:04:25 UTC, Mike Parker wrote:

> https://dlang.org/blog/2017/10/20/unit-testing-in-action/

I'm somewhat late to this party.... but anyway, here is my two 
cents on the way Unit testing needs to be tweaked.

One of the values of Unit Testing is Defect Localization.

ie. In a well designed unit test suite, tell me which test 
failed, I will tell you, to within a few lines, where the bug is.

However in the presence of setup and teardown failures, we lose 
that.

Ideally we should differentiate between failures that occur 
during setup and teardown, versus exceptions occurring in the 
behaviour under test, or assertions on the validity of the result.

ie. Failures under setup and teardown are not failures of the 
behaviour under test. The only thing we can say about the 
behaviour under test in these cases is, that it “hasn’t been 
tested”.

Hopefully the behaviour that failed during the setup and teardown 
is explicitly tested elsewhere.

ie. We should stop at the first test that fails at a point other 
than setup and teardown, as this is likely to be the cause, for 
the cascade of failures in setup and teardown of other tests.