[phobos] Silent failure of std.container unittests

[Jonathan M Davis]

On Friday, July 16, 2010 17:47:33 Benjamin Shropshire wrote:
> 
> Can we all agree that  a failed assert should end a "Unit test" ("Unit
> test" != unittest)?

In any unit test framework that I've used and certainly any place that I've 
worked at which used unit tests, a unit test _is_ a function. A single assertion 
is just part of the test.

> 
> My thought is, that we might be running into issues because the idea of
> a unit test is being conflated with a function. What we really want is a
> collection of blocks of code that that are all run and if any of them
> trigger an assert, that block stops and the tests as a whole are
> considered to have failed.
> 
> As a point to start from, how about allow unittest as a statement type?
> 
> foreach(item; collection) unittest { asssert(foobar(item)); }

Regardless of whether you consider the assertion or the unittest block as the 
unit test (and I would certainly choose the latter), I don't see any point to 
this. What would this do? A unittest block is a block of code which is run 
before main when your program is built with -unittest. How would that jive with 
a foreach loop? Do you want code within functions to suddenly be compiled in or 
not depending on whether the -unittest flag was used?

I think that what we have in terms of code structure is quite good. The only 
suggestion that I would make to that would be to add optional names (e.g. 
unittest(test_name) {}) so that it would be possible at a later date to make it 
so that tests could be run individually (particularly by 3rd party tools like 
IDEs). But that could be added at pretty much any time, since it's not a 
breaking change.

Regardless, I don't understand what your foreach loop is really trying to do 
here. Are you trying to get it to get it to run the test on each item in the 
collection and report errors at each failure without affecting the other tests? 
If that's the case, I don't see what this buys us over just have a special 
function - like expect() or verify() or check() or whatever - which prints out 
the error and continues, having set a global flag to indicate that one of the 
unit tests failed so that main() won't be run.

- Jonathan M Davis