Code coverage in Phobos

[Seb via Digitalmars-d]

On Sunday, 10 July 2016 at 02:38:07 UTC, Seb wrote:
> On Wednesday, 25 May 2016 at 02:34:44 UTC, Seb wrote:
>> On Tuesday, 29 March 2016 at 20:50:57 UTC, Seb wrote:
>>> Okay I see that for the long run we need a better way to 
>>> handle the testing infrastructure :/
>>
>> Actually the idea of achieving "100% coverage" is that we test 
>> every line at least once and don't let this testing by done by 
>> users.
>> So to refresh the discussion - there were two general ideas
>>
>> 1) Find "bad", "dangerous" modules, e.g.
>>
>> /xml.d 64%
>> /zlib.d 60%
>> /experimental/allocator/typed.d 54%
>> /experimental/allocator/building_blocks/segregator.d 50%
>> /experimental/allocator/building_blocks/bucketizer.d 48%
>> /encoding.d 66%
>> /container/binaryheap.d 59%
>> /digest/digest.d 72%
>>
>> 2) Increase coverage for generic, platform-independent modules 
>> like std.algorithm to 100%
>>
>> I know it's a lot of work, but shouldn't that make our jobs us 
>> maintainers easier (=catch the bugs before you have to fix 
>> them?). Therefore I am bumping this ;-)
>
> Just a quick follow-up on this. I finally managed to work a bit 
> on the export to Codecov.io - it's pretty neat as it will warn 
> reviewers if the coverage decreases due to changes in a PR.
> Moreover according to Codecov.io, Phobos currently has an 
> overall coverage of 88.15% (the actual coverage is a bit 
> higher, because I had to disable a few tests for Travis and 
> missing, "uncovered" lines are also reported for e.g. 
> deprecated lines).
>
> So if someone wants to improve not well-tested modules, at [1] 
> is a list of all modules in Phobos with their coverage. Some 
> highlights include:
>
> regex: 51%
> encoding.d: 62%
> mathspecial: 57%
> mmfile: 61%
> process: 69%
> socket: 66%
> zlib: 60%
>
> Hopefully in a few days this will be part of Phobos (see [2] 
> for details) and thus automatically updated ;-)
>
> [1] 
> https://codecov.io/gh/wilzbach/phobos/tree/5fc9eb90076101c0266fb3491ac68527d3520fba/std
> [2] https://github.com/dlang/phobos/pull/4587

Short update - the experiment is now live. Please help to kill 
the uncovered bits in Phobos :)
A short overview of features CodeCov provides:

1) A CodeCov bot will warn the reviewers if (a) the code coverage 
decreased or (b) there are new, unhit lines added within a PR.

2) For every commit one can browse the current code coverage of 
all modules
https://codecov.io/gh/dlang/phobos/tree/77bee525787a90759211dfeb7103ca608bb44bf0/std

3) They provide a handy dashboard with an overview of historic 
information

https://codecov.io/gh/dlang/phobos

(It's a bit empty atm, it might look like this: 
https://codecov.io/gh/libmir/mir)

4) CodeCov has a bunch of other features, for example browser 
extensions that integrate the code coverage report directly onto 
Github diffs:

https://github.com/codecov/browser-extension

That being said the current approach isn't perfect (see the PR 
for details), but it's hopefully a start to pay more attention to 
code coverage ;-)