Review: A new stab at a potential std.unittests [Update]

[Jonathan M Davis]

On Saturday 27 November 2010 09:00:36 Jacob Carlborg wrote:
> On 2010-11-26 09:10, Jonathan M Davis wrote:
> > Any more feedback on my potential std.unittests, or is looking good
> > overall? I definitely think that it's better than when I first posted
> > it, so the feedback thus far has definitely been helpful, and I do find
> > these functions extremely useful in my own code.
> > 
> > - Jonathan M Davis
> > 
> > 
> > P.S. My most recent update for std.datetime does include the most recent
> > version of unittests.d, so if you want to see them in more large scale
> > use, you can look there.
> 
> How about catching AssertErrors (if possible), collecting them and then
> at the end of the unit test output all failed unit tests. This will
> allow the unit tests after a failing unit test to also run.
> 
> Feel free to have a look at my rspec inspired unit test "framework":
> http://dsource.org/projects/orange/browser/orange/test/UnitTester.d

All of my functions work essentially the same as assert does. That means that 
once an assertion fails (or one of my functions fails) in a unittest block, that 
block has failed and is done. This has been discussed before, and the general 
consensus is that this is the correct and desired behavior. It can be a bit 
annoying if you have a whole bunch of unrelated assertions in a single unittest 
block, but if you're doing anything where each assertion relies on state with an 
assertion relying on previous assertions passing, and assertions didn't really 
stop the test from running, you'd get a ton of irrelevant errors after a single 
failure.

My functions are intended to expand on assert, giving you better error messages 
on failures and reducing boiler plate code, not change how unit tests in D 
fundamentally work. And I don't want to introduce stuff into Phobos which changes 
how unit tests in D fundamentally work. I think that it's great as is. It's just 
that helper functions that produce better error messages than assert, and helper 
functions which reduce boiler plate code in common test cases can improve on 
assert and make for better unit tests, and I think that they would be a good 
addition to Phobos.

Currently, if a unittest block fails, then none of the other unittest blocks in 
that module run, but unittest blocks in other modules do run. It's supposed to 
be changed so that all unittest blocks in a module are run even if one fails, 
but that change hasn't been made yet. Once that is made, you'll be able to split 
up your unittest blocks to more thoroughly to allow for more tests to run on a 
single failure if you'd like to. But it is by design that once you have an error 
in a unittest block, that unittest block does not continue to run.

- Jonathan M Davis