If you do something dumb enough, the test runner won't print out your test failures

[Nick Sabalausky via Digitalmars-d]

On 10/23/2016 05:52 PM, Jonathan M Davis via Digitalmars-d wrote:
>
> import core.sys.posix.unistd;
>
> unittest
> {
>      close(2);
>      assert("hello" == "7");
> }
>

Ouch! I like the punchline here :)

> However, it does highlight the fact that if you just look at the output of
> running the unit tests and not the exit code, you might think that your
> tests succeeded when they actually failed. But fortunately, that's not
> terribly likely (much as I managed to do it), and if you're using dub, it
> does check the exit code.
>

Honestly, it's gets worse than that IMO:

I've had a bunch of time over the course of using D, with any build tool 
from bud/xfbuild (remember those?), to rdmd, to dub, where a slip-up in 
my buildscript, project configuration, or a missing import, etc would 
result in one or all of my unittests *silently!* failing to run.

I even hit it again just this week - where most tests in mysql-native 
were running...except for one, because of a missing import. The only 
reason I caught it was because I had learned to be pedantic and 
temporarily tossed in a writeln "just to be ABSOLUTELY sure...". Good 
thing I did.

This example is not only easier to hit than yours, but it's also worse 
because even the exit code will report "A-ok!".

These are all examples of why I was always opposed from the beginning to 
D's default of "no output unless a test fails": Because it makes 
"success" indistinguishable from certain types of failures.

That's why I always tried to add some "Testing blah blah whatever..." 
output at the start of every test in most of my projects: so I *KNOW* 
that what merely looks like unittest success actually *IS* unittest success.

That's also one of the biggest things I like about unit-threaded: It 
does that automatically, AND (IIRC), it shows a count of how many tests 
were run - so when I have any project with more than 
like...five...tests, I can look at that number and *know* that I'm not 
overlooking a "Hey, wait, this list of tests run is missing 'Testing 
fizzbar'!!!"