[phobos] Silent failure of std.container unittests

[Steve Schveighoffer]

>From: David Simcha <dsimcha at gmail.com>
>Vote++.  What made it hit home for me was when I was unittesting some code where 
>
>the unittests can't be completely deterministic and thus print out lots of crap 

>for human interpretation.  (When unittesting random number generators or stress 

>testing a parallelization lib this can be a necessary evil IMHO).  In these 
>cases it's easy for a single assert failure to go unnoticed.
>
>On the other hand, I guess we could start using enforce() instead of assert() in 
>
>unittests if we prefer the old behavior.

Unless you rely on asserts in functions that you call...

But in order for unittests to make sense, you need three things:

1. only failures print out.
2. a unit test stops after a failure occurs.
3. the program does not run main() and returns a non-zero error code if a 
unittest fails.

An optional feature is that when a unit test fails, the next unit test runs.  
The unit test that has a failure should *NOT* continue to run, as it does now.  
Basing tests on invalid state, so basically only the first assert in a unit test 
is valid is a recipe for annoyance.

The expectation is that unit tests should be passing.  If you screw something 
up, a small number of unit tests will start failing, so there is little danger 
of you missing them.

I don't share your view of "losing data" if you are printing extra data during 
unit tests, you can easily pipe the results to a file, and search for the first 
assert.  If you are not doing this, I don't really know why you would be 
printing out so much data in the first place.  In fact, I'd argue that unit 
testing in this way is not a good idea.  Unit tests should not be printing 
anything, if you need to "print debug," you should use a separate program.

-Steve