Helper unit testing functions in Phobos (possible std.unittests)

Jonathan M Davis jmdavisProg at gmx.com
Sat Nov 6 21:05:37 PDT 2010


On Saturday 06 November 2010 18:31:19 Nick Sabalausky wrote:
> "Jonathan M Davis" <jmdavisProg at gmx.com> wrote in message
> news:mailman.139.1289084847.21107.digitalmars-d at puremagic.com...
> 
> On Saturday 06 November 2010 13:05:13 Jérôme M. Berger wrote:
> > spir wrote:
> > > On Sat, 6 Nov 2010 01:21:25 -0700
> > > 
> > > Jonathan M Davis <jmdavisProg at gmx.com> wrote:
> > >> I believe strongly that a unit test block which has a failure should
> > >> end
> > >> excecution. For many such tests, continuing would be utterly
> > >> pointless, since each successive test relies on the last.
> > > 
> > > I don't understand. I can have one dozen test cases for each of one
> > > dozen
> > > funcs. All 144 tests are independant. I prefere the possibility to see
> > > all test errors in one go, if any. Anyway, there may be a flag
> > > STOP_AT_FIRST_TEST_ERROR (or the opposite).
> > 
> > In that case, you should have a dozen dozen different test blocks,
> > each for a single test case of a single function. Under Jonathan's
> > proposal, when a test block fails, it would stop the current *block*
> > (i.e. test case) but allow the remaining blocks to be run.
> 
> If you have a bunch of independent tests, as I typically do, that's just a
> pain in the ass. Even more of a PITA when there's common setup for each of
> them. I find the non-independent tests to be far less common, anyway.
> Besides, even if a few unnecessary asserts are triggered instead of
> skipped, who the hell cares?
> 
> Heck, I went to the bother of creating a unittest library largely just to
> work around the "quit running the tests at the drop of a hat" absurdity.

Every other unit test library that I've used stops running a test when that test 
fails. I have frequently written tests where many of the asserts rely on the 
previous asserts succeeding. You can easily get a _lot_ of cruft in the output 
when you have irrelvant assertions failing. It was essentially agreed upon in 
the last discussion about this on the Phobos list that assert _should_ result in 
the test as a whole failing and that was the _desired_ behavior. The part that 
we're missing right now is having each subsequent unittest block in a module run 
after a single such block fails.

It is, of course, possible to write unit testing functions which print out a 
failure instead of throwing an AssertError, but that goes against how they work 
and would make creating additional unit testing tools (like having on IDE run 
the unit tests and indicate which succeeded and which failed) harder if not 
impossible to do correctly if and when we finally are able to do those (obviously 
various improvements to the unit testing framework in D need to be made first 
however - such as making it possible to name unittest blocks).

Obviously, you can write your unit tests however you think is appropriate, but 
what I'm trying to do here is create extra unit test functions which function as 
assert does except that they print out more information on failures (such as 
indicating what the values being compared were when comparing for equality) and 
reduce error-prone boiler-plate code (such as testing that a specific exception 
was thrown). I'm not trying to change how unittest blocks function.

- Jonathan M Davis


More information about the Digitalmars-d mailing list