Documented unittests & code coverage

[Walter Bright via Digitalmars-d]

On 8/4/2016 12:04 PM, Atila Neves wrote:
> What I take issue with is two things:
>
> 1. Code coverage metric targets (especially if the target is 100%).  This leads
> to inane behaviours such as "testing" a print function (which itself was only
> used in testing) to meet the target. It's busywork that accomplishes nothing.

Any metric that is blindly followed results in counterproductive edge cases. It 
doesn't mean the metric is pointless, however, it just means that "good 
judgment" is necessary.

I don't think anyone can quote me on a claim that 100% coverage is required. I 
have said things like uncovered code requires some sort of credible 
justification. Being part of the test harness is a credible justification, as 
are assert(0)'s not being executed.

Leaving the multi-codepoint Unicode pathway untested probably has no credible 
justification.


> 2. Using the code coverage numbers as a measure of unit test quality. This was
> always obviously wrong to me, I was glad that the research I linked to confirmed
> my opinion, and as far as I know (I'd be glad to be proven wrong), nobody else
> has published anything to convince me otherwise.
>
> Code coverage, as a measure of test quality, is fundamentally broken. It
> measures coupling between the production code and the tests, which is never a
> good idea. Consider:

All that means is code coverage is necessary, but not sufficient. Even just 
executing code and not testing the results has *some* value, in that it verifies 
that the code doesn't crash, and that it is not dead code.

----

One of the interesting differences between D and C++ is that D requires template 
bodies to have valid syntax, while C++ requires template bodies to be both 
syntactically correct and partially semantically correct. The justification for 
the latter is so that the user won't see semantic errors when instantiating 
templates, but I interpret that as "so I can ship templates that were never 
instantiated", a justification that is unsupportable in my view :-)