std.unittests for (final?) review

[Nick Sabalausky]

"Jonathan M Davis" <jmdavisProg at gmx.com> wrote in message 
news:mailman.405.1294111260.4748.digitalmars-d at puremagic.com...
> On Monday 03 January 2011 16:39:49 Nick Sabalausky wrote:
>> 2. The assertOp* functions are an ugly JUnit/NUnit-style kludge. I'd much
>> rather use something that's more like:
>>
>> int x = 2;
>> assert(x == 7, x);
>> // Or
>> assert(q{_==7}, x);
>> // Or
>> assert(q{ #x#==7 });
>> // etc, anything that involves writing a real expression...
>>
>> Output:
>>
>> Failed: [x == 7] x: 2
>>
>> Some of those exact thing can't be done right now, but it's been 
>> suggested
>> that there should be some _traits-ish way to get the actual code sent to 
>> a
>> function's param as a string. That would make a lot of it possible. Even
>> without that, it would also be possible with mixins and q{}. And yes,
>> people whine about that sort of thing being ugly, and I agree, but this 
>> is
>> just another reason to clean mixins up. And even with the mixin() and q{}
>> noise, I'd still vastly prefer it because of the same reason I'm a
>> proponent of operator overloading - I don't ever want to have to write a
>> basic expression in some bizarre "Add(x, y)" style.
>
> Well, it's not possible to overload assert, so that sort of thing really 
> doesn't
> work. You're kind of stuck with something JUnit-ish in terms of names, 
> though
> assertCmp and assertOpCmp both take a string for the operator, so you 
> don't have
> assertLessThan, assertGreaterThan, etc. And what's there uses lazy instead 
> of
> string mixins, because it's cleaner that way. lazy does the job just fine.
>
> And even if you could overload assert, you'd likely run into the same sort 
> of
> problems that come up when you try and overload constructors and the like 
> and
> the parameters aren't different enough to allow you to do all of the 
> overloads
> that you want to do. At least JUnit-like naming is clear, and you're only 
> going
> to use it unit tests, so it's fairly confined. So, if it bothers you, at 
> least it
> doesn't affect your actual program code.
>

The actual name "assert" isn't really what I was getting at. I realize 
that's not usable, and that's fine. What I meant was, for instance, your 
documentatin includes the example:

assertOpBinary!"+"(Foo(5), 7, Foo(12));

(Although it doesn't seem to compile for me...) That's quite a mess compared 
to the following:

assert(Foo(5) + 7 == Foo(12));

That can currently be done, of course, but it's output on failure obviously 
doesn't give as much useful information as assertOpBinary.

I think a middle ground is needed. For instance, with the assert utils in my 
SemiTwist D Tools library, you can do this:

mixin(deferEnsure!(`Foo(12)`, `_ == Foo(5) + 7`));

And the output upon failure still gives you this information:

Expression 'Foo(12)':
Expected: _ == Foo(5) + 7
Actual: 999

And, though I haven't done so yet, that can be generalized to allow it to 
display what Foo(5) evaluated to. Admittedly, there's lots of room for 
improvement in my syntax there, but the point is that you write an actual 
expression and still get all the same useful information.

So, just as one off-the-top-of-my-head idea, your lib could have something 
like:

assertExpression(`#Foo(5)# + 7 == #Foo(12)#`);

The # delimiters (or whatever system of delimiters would make sense) 
indicate what sub-expression should be displayed, ex:

Assert failed: [Foo(5) + 7 == Foo(12)]
    Foo(5): 6
    Foo(12): 12

Or:

assertExpression(q(a + 7 == b}, Foo(5), Foo(12));
// (same output as before)

The whole things may or may not need to be wrapped in a "mixin()", I don't 
know.

Ie, something like that.


>> 3. Without the ability make them non-fatal (and catch & report
>> unexpectedly-thrown exceptions, and optionally "fatalize" after a few of
>> them have been run), I'd never use any of those assert* functions 
>> directly.
>> It's possible they might be useful in the implementation of an alternate
>> unittest lib, though.
>
> Well, like I said, these functions are intended to work the same way that 
> assert
> does. They're not intended to alter how unit testing works in D, and 
> continuing
> a unittest block after an assertion/unit testing function fails would be
> changing how they work. It may be worthwhile to figure out a way to have 
> some set
> of functions which report errors but don't actually throw an AssertError 
> for
> those who really want that sort of thing, but without hooks of some kind 
> in the
> built-in unit testing framework, I don't know how you'd do that very 
> cleanly.
> So, if that sort of functionality is really wanted, changes are going to 
> have to
> be made in the core unit testing stuff (which is implemented in druntime, 
> I
> think, but it may require compiler changes as well).
>

What about a flag that sets whether the AssertError is thrown or 
accumulated? For instance:

// How it (presumably) is right now:

void assertEqual(...)
{
    perform test;
    if(test failed)
    {
        throw new AssertError(all relevant info);
    }
}

// Suggestion:

bool accumulateFailures=false;
AssertError[] failures;

void assertEqual(...)
{
    if(accumulateFailures)
    {
        try
            perform test;
        catch(e)
        {
            failures ~= new AssertError(all relevant info, including e);
            return;
        }
    }
    else
        perform test;

    if(test failed)
    {
        auto err = new AssertError(all relevant info);

        if(accumulateFailures)
            failures ~= err;
        else
            throw err;
    }
}

void fatalize()
{
    if(failures.length > 0)
    {
        throw new AssertError(concat everyting in failures);
    }
}