Re: DMD unittest fail reporting…

[Jacob Carlborg via Digitalmars-d]

On 2015-12-05 12:12, Russel Winder via Digitalmars-d wrote:

> I put it the other way round: why do you want a stack trace from a
> failure of a unit test? The stack trace tells you nothing about the
> code under test that the test doesn't already tell you. All you need to
> know is which tests failed and why. This of course requires power
> asserts or horrible things like assertEqual and the like to know the
> state that caused the assertion fail. For me, PyTest is the model
> system here, along with Spock, and ScalaTest. Perhaps also Catch.

ScalaTest will print a stack trace on failure, at least when I run it 
from inside Eclipse. So will RSpec which I'm guessing ScalaTest is 
modeled after. In RSpec, with the default formatter it will print a dot 
for a passed test and a F for a failed test. Then at the end it will 
print the stack traces for all failed tests.

> Just because some unittests have done something in the past doesn't
> mean it is the right thing to do. The question is what does the
> programmer need for the task at hand. Stack traces add nothing useful
> to the analysis of the test pass or fail.

I guess it depends on how you write your tests. If you only test a 
single function which doesn't call anything else that will work. But as 
soon as the function you're testing calls other functions a stack trace 
is really needed.

What do you do when you get a test failure due to some 
exception/assertion is thrown deep inside some code you have never seen 
before and how no idea how the execution got there?

> I will be looking at dunit, specd and dcheck. The current hypothesis is
> though that the built in unit test is not as good as it needs to be, or
> at least could be.

The built-in runner is so bad it's almost broken.

-- 
/Jacob Carlborg