Rant after trying Rust a bit

Jonathan M Davis via Digitalmars-d digitalmars-d at puremagic.com
Sat Jul 25 02:53:00 PDT 2015


On Saturday, 25 July 2015 at 08:58:26 UTC, Walter Bright wrote:
> It's still unusual to have 100% coverage in Phobos, and this is 
> not because it is hard. Most of the time, it is easy to do. 
> It's just that nobody checks it.

Yeah. I thought there had been a change to make it so that the 
coverage got printed out as part of the build, but I don't see it 
right now, at least on FreeBSD. Folks would still have to pay 
attention to those numbers though. Writing the tests is easy, and 
if someone is conscientious about their tests, I'd expect them to 
typically hit 100% without having to check (though you can still 
miss the occasional branch even then - especially with templated 
code), but frequently folks just write a few tests to make sure 
that the most basic functionality works and then call it a day.

I know that I'm too often guilty of assuming that I hit 100%, 
because I was thorough with my testing (since I tend to be _very_ 
thorough with my tests), and I should do better at verifying that 
I didn't miss something. I did put an effort a while back in 
making sure that std.datetime was as close to 100% as was 
possible, but I haven't checked it in a while...

Well, that's embarrassing. Almost all of the uncovered lines are 
lines that should never be run - e.g. assert(0) - but it does 
look like there are some lines which aren't covered which should 
be (not many in comparison to the whole, but there shouldn't be 
any). Interestingly though, the coverage is worse than it should 
be because it was generated from the release build on the unit 
tests, and stuff like invariants doesn't get run. I'll have to 
figure out how to get it to give me the coverage for the debug 
run of the tests, since that would be more accurate - though 
std.datetime can never actually hit 100% thanks to all of the 
assert(0) lines in it and the scope(failure) lines for printing 
out extra information when a test does fail. I suspect that I'm 
all of a percentage point off of what the max is.

Interestingly enough, @disable this() {} counts as a 0 too, even 
though the compiler should know that it's impossible for that to 
run even if the code is wrong - unlike assert(0). It _would_ be 
nice though if the assert(0) lines at least weren't counted, and 
it is kind of weird that the unit test lines count (though aside 
from scope(failure) lines, those should all run, though since I 
tend to put scope(failure) lines in unit tests for better output 
on failures, that's going to hurt my code coverage). So, 
_actually_ hitting 100% is likely to be annoyingly elusive for a 
lot of code, even if it's actually fully tested. But while 
std.datetime is almost as close as it can get, it's still 
_almost_ as close as it can get rather than all the way. :(

Clearly, I have a PR or two to write...

> Although we have succeeded in making unit tests part of the 
> culture, the next step is 100% coverage.

Agreed.

> I know that 100% unit test coverage hardly guarantees code 
> correctness. However, since I started using code coverage 
> analyzers in the 1980s, the results are surprising - code with 
> 100% test coverage has at LEAST an order of magnitude fewer 
> bugs showing up in the field. It's surprisingly effective.

Oh, definitely. But while 100% unit test coverage is a huge step 
forward, I also think that for truly solid code, you want to go 
beyond that and make sure that you test corner cases and the 
like, test with a large enough variety of types with templates to 
catch behavioral bugs, etc. So, I don't think that we want to 
stop at 100% code coverage, but we do need to make sure that 
we're at 100% first and foremost.

> This is a huge reason why I want to switch to ddmd. I want to 
> improve the quality of the compiler with unit tests.

That would definitely be cool.

> The various unit tests schemes I've tried for C++ are all ugly, 
> inconvenient, and simply a bitch. It's like trying to use a 
> slide rule after you've been given a calculator.

Yeah. It's not that hard to write one, and the tests themselves 
generally end up being pretty much the same as what you'd have in 
D, but there's still an annoying amount of boilerplate in getting 
them declared and set up - which is annoying enough in and of 
itself, but yeah, once you're used to D's unit tests, it seems 
particularly onerous.

> (I remember the calculator revolution. It happened my freshman 
> year at college. September 1975 had $125 slide rules in the 
> campus bookstore. December they were at $5 cutout prices, and 
> were gone by January. I never saw anyone use a slide rule 
> again. I've never seen a technological switchover happen so 
> fast, before or since.)

If only folks thought that D's advantages over C++ were that 
obvious. ;)

- Jonathan M Davis


More information about the Digitalmars-d mailing list