Unit testing with asserts: Why is assertHandler required to throw?

[Lutger]

On 01/31/2010 09:39 PM, Trip Volpe wrote:
> I recently began porting an existing C++ project of mine (a compiler/interpreter for a dynamic language) to D. In the process I found that the built-in unit testing support, while an awesome concept, was a little bit sparse. In particular, assert() is fairly useless for unit tests, since it throws on a failure and prevents all subsequent tests from running.
>
> Unfortunately, it's the only facility that seems to give you access to the current file and line number, very important bits of info for unit tests in a any non-trivial project. I was contemplating just abandoning D and sticking with C++ and Googletest, but after doing a bit of digging through the source, I found setAssertHandler() in core.exception. Perfect! I could use my own assert handler that just records the error and allows future asserts to still be reached. So I wrote it up and gave it a try, but pretty quickly started getting access violation errors.
>
> I did a bit of searching, and found a bug report on the issue:
>
> http://d.puremagic.com/issues/show_bug.cgi?id=3208
>
> In the comments it's indicated that this is intentional, that the compiler expects the assertion handler to throw. So I guess I have a few questions:
>
> 1. Why is this the expected behavior? It seems to me that there are relatively few useful things you can do with a custom assert handler unless it is possible to refrain from throwing.
>
> 2. Is this going to be fixed any time soon? It seems fairly important to me; it really kills the value of built-in unit tests to be forced to choose between not having any line number information and being able to recognize only one assert failure per test run(!!!).
>
> 3. Are there any current workarounds for the problem? I could just use my own assertion function, but 1) I can't call it "assert" because that's a reserved word, and 2) most importantly, it won't have any way of indicating in its diagnostic output which source line the failure occurred on. In C++, of course, I could use preprocessor macros to do that, but D omits a preprocessor (for very good reasons). Unfortunately, this seems like a case where D hasn't provided replacement functionality.
>
> But maybe there's a sneaky fix I haven't thought of? I also found Runtime.moduleUnitTester() in core.runtime, which is very useful in itself, but provides an _almost_ useful improvement: I can catch the AssertErrors on a module-by-module basis, which would at least allow the unit tests for all modules to be run regardless of a failure in one of them. However, a single module could easily have dozens of tests in it, each of which could contain many individual asserts. Stopping the test early because one assert failed makes no sense.
>
> It would be _almost_ acceptable to at least be sure of running all the tests in each module; is there any way to poke inside or otherwise override ModuleInfo's unitTest() function?
>
>
> Thanks in advance for any advice or comments! D is a completely awesome language, but I found this issue a bit strange.


You can use line and file info with default arguments, this works 
(special case). I just hack around the default unittest system, 
something like this:

void test(string testName)(void delegate () testClosure,
                            int line = __LINE__,
                            string file = __FILE__)
{
     // register test start, run testClosure, end test
}

void expectEquals(A, B)(A a, B b,
                         int line = __LINE__,
                         string file = __FILE__)
{
     // record outcome of assertion
}

unittest
{
     test!"truth of it"( {
         expectEquals(true, true);
     } );
}

or slightly abusive:

unittest
{
     test!"truth of it" =  {
         expectEquals(true, true);
     };
}