std.unittests for (final?) review

[Michel Fortin]

On 2011-01-06 11:34:48 -0500, Jonathan M Davis <jmdavisProg at gmx.com> said:

> On Thursday 06 January 2011 05:30:56 Michel Fortin wrote:
>> There's an other issue that's bothering me with these assertion
>> functions... with 'assert', assertions behaves differently depending on
>> where you write them. In regular code, on failure it calls _d_assert*,
>> in a unit test on failure it calls _d_unittest*, and in contracts for
>> virtual function... well that case I don't understand it fully but it
>> does something very special to implement contract inheritance.
>> 
>> What does assertPred do? It does "throw new AssertError(...)",
>> irrespective of the context. I'm pretty sure that'll break contract
>> inheritance if used inside a contract. To be truly a replacement for
>> assert, assertPred would need to know in which context it is called and
>> generate the appropriate code, although I somewhat doubt it is even
>> possible to generate the right code for contracts without the compiler
>> doing it.
> 
> I don't know anything about this. As far as I know, there's no 
> difference between
> assert in unittest blocks and assert in contracts. That being said, you're
> probably more knowledgeable than I am about that.
> 
> I would have thought was that all it would take would be for the AssertError to
> be handled appropriately by whatever catches it. In the case of contracts, I
> would think that it's an issue of the right contract code being called in the
> right order, and that which contract threw the test would be irrelevant (the
> stack track would show where it was; all that matters from the runtime's
> perspective is that there _was_ an AssertError thrown and that execution is
> therefore going to be stopping).

The thing is that for 'in' contracts you need to have *one* contract 
work in the inheritance chain. So if a contract fails it should just 
check the inherited contract, and if it fails it continues like that. 
If all the inherited contract fails then only then you must throw. I'm 
not sure exactly how DMD generates the code, but I'm pretty sure it 
doesn't throw then catch then throw then catch until it reaches the 
last inherited contract (that'd be unacceptably slow), so if you do 
throw in the contract then you're probably breaking how it works.

That said, I'm pretty sure this only apply to 'in' contracts. Other 
contracts don't exhibit this all-must-fail-to-fail behaviour.

Here's a test case:

	class A {
		// accepts i == 0
		void fun(int i)
		in { assert(i == 0); }
		body {}
	}
	class B : A {
		// accepts i == 0 || i == 1
		override void fun(int i)
		in { assert(i == 1); }
		body {}
	}

	B b = new B;
	b.fun(1);
	b.fun(0);
	b.fun(-1); // only this one breaks the contract


> As for unittest blocks, I thought that it caught the AssertError from the
> unittest block and dealt with it. If it does that, then I don't see why 
> it would
> need a special version of assert. I know that it _used_ to be 
> different, because
> Walter temporarily made it so that within unittest blocks assert set a flag
> saying that the test failed and printed the error rather than throwing an
> AssertError, but Andrei and others complained about that (both that 
> assert would
> have different behavior in different places and that a unittest block would
> continue after a failure), and it was agreed that assert would throw an
> AssertError like it normally does.

The code it generates is different: it calls different assertion 
failure handler in the runtime (_d_assert and derivatives vs. 
_d_unittest and derivatives). But druntime was later changed so that 
both handlers do the same thing (except for different default error 
messages). As long as druntime makes them do the same thing, there 
won't be a problem.

Should there be a difference between the two? This difference could be 
used to distinguish failed tests from failed assertions in function the 
test is calling, but I'm not too sure how useful that'd be.


> So, I don't know if assert does something different depending on where it is
> called. The only special case for assert that I'm aware of is assert(0), which
> becomes the halt instruction with -release rather than going away. If 
> there is a
> difference, then we probably need to understand what it is and what issues it
> could cause. Ideally, there wouldn't be any difference.

assert(0) is one special case, assert(object) is another -- it calls 
the object's invariant. The rest is undocumented as far as I know.


> However, I _have_ been using these functions in unit tests, and they work fine
> there. So, as far as unit testing goes, they work. I have _not_ been using them
> in contracts. I created them specifically with unit testing in mind 
> (and in fact,
> with the current code, the entire module is in a version(unittest) block). It
> sounds like there are some folks (including Andrei) who think that it should be
> useable in normal contracts just like assert is. That's simple to fix 
> by removing
> the version(unittest) block, but I don't know if there are any issues with
> throwing an AssertError from a function called within a contract rather than
> asserting directly inside a contract. If there is, I would think that that's a
> more general problem. I've been doing that all the time with invariants (just
> not with any of my unit testing functions), and that's worked as far as I can
> tell, but I've been using structs primarily, which wouldn't have contract
> inheritance. So, I think that assert should _definitely_ work normally when
> called from a function called from a contract rather than when used directly in
> a contract, but I don't know that that never causes problems. We need someone
> who actually knows what assert does in each case to say whether there's an
> issue, I think.

As things stands now, I'm pretty sure the issue only apply if you use 
one of your assert derivative in 'in' contracts (see test case above). 
But I think we need to have Walter involved in this thread to tell us 
where and when and how we can use custom assert functions in place of 
'assert' without breaking things.


-- 
Michel Fortin
michel.fortin at michelf.com
http://michelf.com/