Review: A new stab at a potential std.unittests [Update]

[Jonathan M Davis]

On Saturday 27 November 2010 15:30:23 Jacob Carlborg wrote:
> On 2010-11-27 21:36, Jonathan M Davis wrote:
> > On Saturday 27 November 2010 09:00:36 Jacob Carlborg wrote:
> >> On 2010-11-26 09:10, Jonathan M Davis wrote:
> >>> Any more feedback on my potential std.unittests, or is looking good
> >>> overall? I definitely think that it's better than when I first posted
> >>> it, so the feedback thus far has definitely been helpful, and I do find
> >>> these functions extremely useful in my own code.
> >>> 
> >>> - Jonathan M Davis
> >>> 
> >>> 
> >>> P.S. My most recent update for std.datetime does include the most
> >>> recent version of unittests.d, so if you want to see them in more
> >>> large scale use, you can look there.
> >> 
> >> How about catching AssertErrors (if possible), collecting them and then
> >> at the end of the unit test output all failed unit tests. This will
> >> allow the unit tests after a failing unit test to also run.
> >> 
> >> Feel free to have a look at my rspec inspired unit test "framework":
> >> http://dsource.org/projects/orange/browser/orange/test/UnitTester.d
> > 
> > All of my functions work essentially the same as assert does. That means
> > that once an assertion fails (or one of my functions fails) in a
> > unittest block, that block has failed and is done. This has been
> > discussed before, and the general consensus is that this is the correct
> > and desired behavior. It can be a bit annoying if you have a whole bunch
> > of unrelated assertions in a single unittest block, but if you're doing
> > anything where each assertion relies on state with an assertion relying
> > on previous assertions passing, and assertions didn't really stop the
> > test from running, you'd get a ton of irrelevant errors after a single
> > failure.
> 
> This how it works in my unit test library:
> 
> For every test you call a testing function that takes a delegate, in
> this delegate you have your assertions. This function will call the
> delegate and catch all assert exceptions/errors and collected them for
> later outputting. Now this delegate will end execution if an exception
> is thrown and the remaining code in the delegate will not be run.
> 
> If you want your test to stop after a failing assert you put all the
> asserts into one call to the testing function. If you want your test to
> continue after a failing assert you call the testing function multiple
> times.

I confess that I don't see any real difference between that and simply splitting 
up unittest blocks (assuming, of course, that subsequent unittest blocks in a 
module get run after one fails, which hasn't been implmented yet), and the 
delegate solution at least sounds more complicated.

> > My functions are intended to expand on assert, giving you better error
> > messages on failures and reducing boiler plate code, not change how unit
> > tests in D fundamentally work. And I don't want to introduce stuff into
> > Phobos which changes how unit tests in D fundamentally work. I think
> > that it's great as is. It's just that helper functions that produce
> > better error messages than assert, and helper functions which reduce
> > boiler plate code in common test cases can improve on assert and make
> > for better unit tests, and I think that they would be a good addition to
> > Phobos.
> 
> I guess your functions are like assert helpers and my library is at a
> higher level.

That's probably a good way to put it. I think that higher level libraries 
certainly have their place, but I don't think that it necessarily makes sense to 
put that sort of thing in Phobos.

- Jonathan M Davis