Extending unittests [proposal] [Proof Of Concept]

[Jonathan M Davis]

On Thursday, September 20, 2012 22:55:23 Jens Mueller wrote:
> You say that JUnit silently runs all unittests before the first
> specified one, don't you?

Yes. At least, that was its behavior the last time that I used it (which was 
admittedly a few years ago).

> If that is done silently that's indeed strange.

It could have been a quirk of their implementation, but I expect that it's to 
avoid issues where a unit test relies on previous unit tests in the same file. 
If your unit testing functions (or unittest blocks in the case of D) have 
_any_ dependencies on external state, then skipping any of them affects the 
ones that you don't skip, possibly changing the result of the unit test (be it 
to success or failure).

Running more unittest blocks after a failure is similarly flawed, but at least 
in that case, you know that had a failure earlier in the module, which should 
then tell you that you may not be able to trust further tests (but if you 
still run them, it's at least then potentially possible to fix further failures 
at the same time - particularly if your tests don't rely on external state). 
So, while not necessarily a great idea, it's not as bad to run subsequent 
unittest blocks after a failure (especially if programmers are doing what 
they're supposed to and making their unit tests independent).

However, what's truly insane IMHO is continuing to run a unittest block after 
it's already had a failure in it. Unless you have exceedingly simplistic unit 
tests, the failures after the first one mean pretty much _nothing_ and simply 
clutter the results.

> When has this been merged? It must have been after v2.060 was released.
> Because I noticed some number at the end of the unittest function names.
> But it was not the line number.

A couple of weeks ago IIRC. I'm pretty sure that it was after 2.060 was 
released.

- Jonathan M Davis