Assert and the optional Message

[Steven Schveighoffer]

On Sat, 10 Mar 2012 11:03:43 -0500, Jacob Carlborg <doob at me.com> wrote:

> On 2012-03-09 20:06, Jonathan M Davis wrote:
>
>> In what way? Yes, they're _catchable_, but everything that was on the  
>> unwound
>> portion of the stack is now in an undefined state. So, recovering from  
>> the
>> AssertError and continuing execution doesn't work. So, a handler can  
>> catch the
>> AssertError and do something before letting it kill the program, but  
>> portions
>> of the program are definitely in an undefined state when an AssertError  
>> is
>> caught. The closer the catch point is to the throw point, the less of  
>> an issue
>> that is, but unless you can guarantee that no destructors, scope  
>> statements,
>> or finally blocks were between the throw point and the catch point,  
>> then some
>> portion of the program is in an undefined state, and continuing program
>> execution is a bad idea. It _is_ possible to catch an AssertError, but  
>> you
>> have to be very careful about what you do after that.
>>
>> I'm not aware of anything in contracts which would involve catching an
>> AssertError and then continuing execution. When a contract fails, it's  
>> going
>> to kill the program just like any other assertion failure does. So, I'm  
>> not
>> quite sure what you're referring to.
>>
>> - Jonathan M Davis
>
> This might be how it works today, but I think it's a broken design.  
> AssertError MUST be SAFELY catchable to be able to write unit testing  
> frameworks.

There was a period of time where unit test asserts were handled  
differently than normal asserts.  The unit tests would continue even if a  
unit test asserted.  This is no longer the case.

The issue was that sometimes when you have one small problem in your code,  
you get thousands of asserts from different unit tests, making it  
impossible to find the "real issue".

I contended that unit test asserts should result in a hook that by default  
kills the program, but could be hooked into a unit testing handler.  But  
the consensus was just to switch back to the original behavior (all  
asserts kill the program).  If you look through the phobos or druntime  
mailing list archives you might find the relevant conversations.  It may  
even be so old that the archives don't have it (it was on a private  
mailing list on Andrei's server for a time).

My thoughts on assert/unit test:

1. AssertError (and any other Error) should not be caught by normal user  
code.  Ever.  It should only be caught by the runtime.  However, the  
handler in the runtime could provide a hook specific to the OS/application  
and whether it is running unit tests.  Obviously the hook would have to be  
set up before main runs, so that is an issue, but you may just need to  
alter the runtime in that case.
2. Any assert in a unit test block should halt execution in the unit test  
block.  Often subsequent asserts are just noise because you are depending  
on the conditions that make the prior asserts pass.
3. unit test blocks should be tested individually, not on a module-level  
IMO.  I'm not sure if this currently is the case.
4. Jonathan's statement that the program is in an "undefined state" is not  
quite true.  It's in an unwound state, but not undefined.  Certainly, you  
should be able to continue running more unit tests.  Unit tests should be  
a) unaffected by other unit test blocks, and b) should not affect the  
running of any other code in any way.  Otherwise, a program would run  
differently depending on whether unit tests ran.
5. If *any* unit tests fail, the main() function should not be run.  If an  
assert is triggered while running main(), the program should exit.

-Steve