Unit tests and segfaults

[Jonathan M Davis]

On Tuesday, December 11, 2012 17:16:58 Russel Winder wrote:
> I am not relying on segfaults, that would just be silly ;-) The
> issue is that unit tests should test error as well as success. I
> want to know if I get a segfault when I have an infinite
> recursion in an algorithm (due to incorrect parameters slipping
> through data validation).
> 
> I am also trying to investigate tail recursion optimization and
> getting a segfault is a way of detecting that it isn't happening.

I'd argue that if you want an error condition that you test for, you should 
actually do something in your code that generates an error condition (e.g. 
throw an exception) and that it really makes no sense to test for segfaults. 
That's not really an error condition. That just means that your code is 
broken. And such cases aren't generally worth testing for IMHO.

You test that a function works correctly with various inputs, but if it's 
given something that it's not supposed to accept, and you're not going to 
throw or assert that it's acceptable input, then it doesn't really matter what 
happens. Either you're doing defensive programming and checking inputs (in 
which case, your unit tests will catch error cases that don't fail properly 
when the exception isn't thrown), or you're doing design by contract, in which 
case, it doesn't matter what happens when the caller passes invalid input, 
because they've violated the contract. The unit tests on the caller should 
then catch that as opposed to the unit tests on the function being called. 
Testing that the function doesn't do weird things when the contract is broken 
is a waste of time. If you want to avoid weird stuff happening when contracts 
are broken, then use defensive programming and protect the function against 
such bad input.

- Jonathan M Davis