Documented unittests & code coverage

[Johannes Pfau via Digitalmars-d]

Some time ago we moved some example code from documentation comments
into documented unittests. Some of these more complicated examples are
incomplete and therefore are not expected to actually run. These are
also not very useful as real unittests as they do not contain any
asserts or verification of results. Making them documented
unittests was mainly done to make sure these examples keep compiling
when there are library changes.

I just had a quick look at https://github.com/dlang/phobos/pull/4587
and some example output:
https://codecov.io/gh/wilzbach/phobos/src/5fc9eb90076101c0266fb3491ac68527d3520fba/std/digest/digest.d#L106

And it seems that this 'idiom' messes up code coverage results: The
code in the unittest is never executed (we just want syntactically
valid code) and therefore shows up as untested code. The code coverage
shows 46 missed lines for std.digest.digest, but only 8 of these
actually need testing.

So how do we solve this?
* Ignore lines in documented unittests for code coverage?
* Make these examples completely executable, at the expense of
  documentation which will then contain useless boilerplate code
* Move these examples back to the documentation block?


And as a philosophical question: Is code coverage in unittests even a
meaningful measurement? We write unittests to test the library code. But
if there's a line in a unittests which is never executed, this does not
directly mean there's a problem in library code, as long as all library
code is still tested. It may be an oversight in the unittest in the
worst case, but how important are ovesights / bugs in the unittests if
they don't affect library code in any way?