[phobos] Silent failure of std.container unittests

[Jason Spencer]

----- Original Message ----
> From: Sean Kelly <sean at invisibleduck.org>

> This seems  a lot like compiler error handling.  A compiler can flag an error 
>and  continue processing, but every successive error is less meaningful than the  
>last.  

That's true because the semantics of the later part of the program depend 
heavily on the former part--how many braces, semi-colons, etc.

But a unit test block for the minus operation on a class Rational may have 
little or nothing to do with the unit test for division in Rational, say.  I 
think what Jonathan is saying is that it's fairly common practice to break up 
unit tests even for a single class into small sections--one for ctors, one for 
operation1, one for operations2 and 3 (because they're related), etc.  In this 
case, you'd like all 3 unit tests for the class to run, even though the new ctor 
you just added broke the first unit test block.  

Having used several of the UT frameworks out there, my own expectation is that 
if I have 5 UT blocks in a module, when I run with -unittest, all 5 blocks will 
run.  If one or more should fail, then main will not run.  The failing test 
block will stop executing at an assertion failure and the next UT will be 
started.  But all 5 would run everytime.

>How about this... since unit tests are run  after static ctors are run, the 
>behavior of whether to throw an AssertError or  report and continue can be a 
>run-time configurable option.  I'd still like  to get more granular tests 
>though, since throwing on error is far less useful if  all unit tests for a 
>module are grouped together rather than run  separately.

I think that's probably the only way to make everyone happy.  That's why I 
suggested making a global UT status var available or allowing a function to be 
defined that could decide to continue or something.  Just have to justify the 
cost.