[phobos] Silent failure of std.container unittests

[Andrei Alexandrescu]

On 07/14/2010 08:58 PM, Walter Bright wrote:
>
>
> Sean Kelly wrote:
>>
>>
>> How about this... since unit tests are run after static ctors are run,
>> the behavior of whether to throw an AssertError or report and continue
>> can be a run-time configurable option.
>
> Frankly, I don't think a configurable option is a good idea. Druntime
> has a lot of user configurability, and as far as I can tell, absolutely
> none of it has ever been used. It just adds complexity, both to the code
> and the documentation.

I agree. But also simple does not mean crappy. I think there's too much 
talk around a simple matter:

======
Assertion failure should abort the current unittest block.
======

Do we all agree about the above? We keep on discussing how the 
usefulness and informativeness of assertion failures falls off a cliff 
after the first failure, and I can't seem to hear loud and clear that we 
all want this same thing.

So we've been through these steps:

1. Crappy: failure to assert causes the program to abort.

2. Awful: assert is hacked inside unittests to essentially be writeln, 
all unittests run regardless. Programs claims success upon exit.

3. Mediocre: assert continues to be hacked inside unittests to 
essentially be writeln + set a flag. Programs exit with errors if the 
flag was set.

NOT YET:

4. DESIRED: assert is NOT hacked, any failing assert ends the current 
unittest, the failure message is printed, execution continues with the 
next unittest, program ends with error code if at least one assert 
failed, everybody's happy.

I'm glad we made progress from 2 to 3. Could you all please share your 
opinion about the delta between 3 and 4?


Andrei