Rant after trying Rust a bit
Jonathan M Davis via Digitalmars-d
digitalmars-d at puremagic.com
Sat Jul 25 02:53:00 PDT 2015
On Saturday, 25 July 2015 at 08:58:26 UTC, Walter Bright wrote:
> It's still unusual to have 100% coverage in Phobos, and this is
> not because it is hard. Most of the time, it is easy to do.
> It's just that nobody checks it.
Yeah. I thought there had been a change to make it so that the
coverage got printed out as part of the build, but I don't see it
right now, at least on FreeBSD. Folks would still have to pay
attention to those numbers though. Writing the tests is easy, and
if someone is conscientious about their tests, I'd expect them to
typically hit 100% without having to check (though you can still
miss the occasional branch even then - especially with templated
code), but frequently folks just write a few tests to make sure
that the most basic functionality works and then call it a day.
I know that I'm too often guilty of assuming that I hit 100%,
because I was thorough with my testing (since I tend to be _very_
thorough with my tests), and I should do better at verifying that
I didn't miss something. I did put an effort a while back in
making sure that std.datetime was as close to 100% as was
possible, but I haven't checked it in a while...
Well, that's embarrassing. Almost all of the uncovered lines are
lines that should never be run - e.g. assert(0) - but it does
look like there are some lines which aren't covered which should
be (not many in comparison to the whole, but there shouldn't be
any). Interestingly though, the coverage is worse than it should
be because it was generated from the release build on the unit
tests, and stuff like invariants doesn't get run. I'll have to
figure out how to get it to give me the coverage for the debug
run of the tests, since that would be more accurate - though
std.datetime can never actually hit 100% thanks to all of the
assert(0) lines in it and the scope(failure) lines for printing
out extra information when a test does fail. I suspect that I'm
all of a percentage point off of what the max is.
Interestingly enough, @disable this() {} counts as a 0 too, even
though the compiler should know that it's impossible for that to
run even if the code is wrong - unlike assert(0). It _would_ be
nice though if the assert(0) lines at least weren't counted, and
it is kind of weird that the unit test lines count (though aside
from scope(failure) lines, those should all run, though since I
tend to put scope(failure) lines in unit tests for better output
on failures, that's going to hurt my code coverage). So,
_actually_ hitting 100% is likely to be annoyingly elusive for a
lot of code, even if it's actually fully tested. But while
std.datetime is almost as close as it can get, it's still
_almost_ as close as it can get rather than all the way. :(
Clearly, I have a PR or two to write...
> Although we have succeeded in making unit tests part of the
> culture, the next step is 100% coverage.
Agreed.
> I know that 100% unit test coverage hardly guarantees code
> correctness. However, since I started using code coverage
> analyzers in the 1980s, the results are surprising - code with
> 100% test coverage has at LEAST an order of magnitude fewer
> bugs showing up in the field. It's surprisingly effective.
Oh, definitely. But while 100% unit test coverage is a huge step
forward, I also think that for truly solid code, you want to go
beyond that and make sure that you test corner cases and the
like, test with a large enough variety of types with templates to
catch behavioral bugs, etc. So, I don't think that we want to
stop at 100% code coverage, but we do need to make sure that
we're at 100% first and foremost.
> This is a huge reason why I want to switch to ddmd. I want to
> improve the quality of the compiler with unit tests.
That would definitely be cool.
> The various unit tests schemes I've tried for C++ are all ugly,
> inconvenient, and simply a bitch. It's like trying to use a
> slide rule after you've been given a calculator.
Yeah. It's not that hard to write one, and the tests themselves
generally end up being pretty much the same as what you'd have in
D, but there's still an annoying amount of boilerplate in getting
them declared and set up - which is annoying enough in and of
itself, but yeah, once you're used to D's unit tests, it seems
particularly onerous.
> (I remember the calculator revolution. It happened my freshman
> year at college. September 1975 had $125 slide rules in the
> campus bookstore. December they were at $5 cutout prices, and
> were gone by January. I never saw anyone use a slide rule
> again. I've never seen a technological switchover happen so
> fast, before or since.)
If only folks thought that D's advantages over C++ were that
obvious. ;)
- Jonathan M Davis
More information about the Digitalmars-d
mailing list