[phobos] Silent failure of std.container unittests

[Lars Tandle Kyllingstad]

On Wed, 2010-07-14 at 21:42 -0500, Andrei Alexandrescu wrote:
> On 07/14/2010 08:58 PM, Walter Bright wrote:
> >
> >
> > Sean Kelly wrote:
> >>
> >>
> >> How about this... since unit tests are run after static ctors are run,
> >> the behavior of whether to throw an AssertError or report and continue
> >> can be a run-time configurable option.
> >
> > Frankly, I don't think a configurable option is a good idea. Druntime
> > has a lot of user configurability, and as far as I can tell, absolutely
> > none of it has ever been used. It just adds complexity, both to the code
> > and the documentation.
> 
> I agree. But also simple does not mean crappy. I think there's too much 
> talk around a simple matter:
> 
> ======
> Assertion failure should abort the current unittest block.
> ======
> 
> Do we all agree about the above?

Yes.


> We keep on discussing how the 
> usefulness and informativeness of assertion failures falls off a cliff 
> after the first failure, and I can't seem to hear loud and clear that we 
> all want this same thing.
> 
> So we've been through these steps:
> 
> 1. Crappy: failure to assert causes the program to abort.
> 
> 2. Awful: assert is hacked inside unittests to essentially be writeln, 
> all unittests run regardless. Programs claims success upon exit.
> 
> 3. Mediocre: assert continues to be hacked inside unittests to 
> essentially be writeln + set a flag. Programs exit with errors if the 
> flag was set.
> 
> NOT YET:
> 
> 4. DESIRED: assert is NOT hacked, any failing assert ends the current 
> unittest, the failure message is printed, execution continues with the 
> next unittest, program ends with error code if at least one assert 
> failed, everybody's happy.
> 
> I'm glad we made progress from 2 to 3. Could you all please share your 
> opinion about the delta between 3 and 4?

I agree that 4 is the best solution.  It is very bad that assert has
different semantics depending on where it's used.  Also, it makes a lot
of sense to consider a unittest{} block to be a single unit test.

-Lars