[phobos] Silent failure of std.container unittests

[Jonathan M Davis]

On Saturday 17 July 2010 10:52:14 Michel Fortin wrote:
> Le 2010-07-17 à 10:10, Jonathan M Davis a écrit :
> > On Saturday 17 July 2010 05:17:09 Michel Fortin wrote:
> >> As other mentioned, what I do here with nested unit tests could easily
> >> be implemented by offering the user a new library function too. I just
> >> happen to think what I'm proposing here is more elegant.
> > 
> > I'd have to argue with you on that one. Personally, I find it to be far
> > uglier than simply replacing assert with expect in cases where you don't
> > want the test to stop on failure. A unittest block without braces is a
> > bit odd, but I could see that working and not being a big deal. But a
> > unit test block inside another? Sure, you could do it, but I think that
> > it would be confusing to new users. It's also more verbose than simply
> > changing assert to expect, and I think that it's a lot uglier.
> > Obviously, this is at least somewhat subjective, however, since you
> > clearly think that nesting unittest is more elegant.
> 
> Well, I did believe it was more elegant a few hours ago while looking into
> it. Now I'm more ambivalent: both are fine with me if the library function
> has a good name which does not clash easily while being short enough. I do
> think 'check' or 'verify' are too clash-prone. Perhaps 'expect' would be a
> better choice.
> 
> > Not to mention, I am a proponent of the idea of named unit tests (e.g.
> > unittest(test_name) {}). And they don't really make sense when nested.
> > Sure, you could have named external ones and non-named nested ones, but
> > then you'd have to worry about whether a test was nested or not when
> > allowing a name on it.
> 
> Assuming we add one day the capability to give a name to a module-level
> unittest, why couldn't we do the same for a nested unittest?
> 
> Note that the implementation I'm proposing (the __doUnitTest function)
> would make it very easy to add extra features to unittest because the
> compiler could simply insert a call to __doUnitTest with more arguments
> (such as a name) and the runtime would do what it wants with it. This
> works both with nested and non-nested unit tests.

You could, but then it wouldn't make sense to run the unit tests individually - 
which is one of the main reasons for having named unit tests. A typical example 
when using JUnit with eclipse, for instance, would be to run all of the unit 
tests in a class as a group. One fails, so you look at the output for that 
specific unit test, fix it, and then run than specific unit test. Having the name 
makes it easier to keep track of which tests need fixing, and it can be a big 
time-saver when running all of the unit tests take a while.

It would make no sense to run a nested unit test however. The outer unit test 
would have to be run in order to run the inner unit test. And while you could 
use the name of the inner unit test for the purpose of reporting which specific 
test failed (as opposed to just the file and line number from the assertion), you 
still can't necessarily nail down which test failed because it could be in a 
loop. You know that _one_ of the loop's iterations failed, but the test name 
doesn't help you any more than the assertion's file and line number do.

So, you could have named, nested unit tests, but the name wouldn't do you much 
good. And really, coming from a background where a unit test is synonomous with 
a function, the idea of nesting unit tests like that makes no sense. I really 
don't think that it gains you anything to make unittest blocks nestable over 
creating a non-throwing assertion function. And if we're really worried about 
name conflicts, we should just prefix any such function with something like test 
or unittest. Personally, I suggested assert_nothrow(), but no one seems to have 
liked that one. I do agree that check() and verify() seem too likely to conflict 
with stuff in user code though.

- Jonathan M Davis