[phobos] Silent failure of std.container unittests

[Sean Kelly]

On Jul 14, 2010, at 5:00 PM, Jonathan M Davis wrote:

> On Wednesday, July 14, 2010 16:30:58 Walter Bright wrote:
>> Jonathan M Davis wrote:
>>> On the other hand, never halting on assertion failures makes for a lot
>>> of cruft in your results.
>> 
>> I don't really understand why that's a problem, or causes extra work.
>> 
>> But I can see that having to divide up the tests into multiple unittest
>> blocks adds extra work.
>> _______________________________________________
>> phobos mailing list
>> phobos at puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/phobos
> 
> Well, I find it perfectly normal having to divide up the unit tests since every 
> unit testing framework that I've used divides up unit tests into functions where 
> each function is a test that succeeds or fails. There may be many assertions in 
> a particular function, but once one fails, that function ceases to run 
> (generally because an exception was thrown which is then caught by the 
> framework). So, it actually makes more sense to me to divide up tests into unit 
> test blocks. True, if you're testing a set of small but unrelated stuff, it could 
> make sense to put them all together in one test, but if you're testing blocks as 
> a whole, then you have the choice of whether to break them up or not just like 
> in another framework, you'd have the choice of whether you have a single 
> function or multiple functions.
> 
> The cruft is a problem because it makes it harder to find the errors you actually 
> care about.

This seems a lot like compiler error handling.  A compiler can flag an error and continue processing, but every successive error is less meaningful than the last.  If the first error was from a missing paren or curly brace, things get really weird really fast.  How about this... since unit tests are run after static ctors are run, the behavior of whether to throw an AssertError or report and continue can be a run-time configurable option.  I'd still like to get more granular tests though, since throwing on error is far less useful if all unit tests for a module are grouped together rather than run separately.