SQLite testing procedures

[Jonathan M Davis]

On Friday 05 August 2011 21:13:17 Robert Clipsham wrote:
> This is a fairly old document now, but it has some interesting ideas
> that could have a place in D (either the language, tools, or phobos):
> 
> http://www.sqlite.org/testing.html
> 
> Some notable things other than the range of testing methods used:
>   * 100% branch test coverage (7.1)
>      - It would be nice if dmd's -cov offered a way to give the coverage
> for statements and branches rather than just per line

But each branch of code is on a particular branch, so if every branch is hit, 
every line is hit. Do you mean cases such as where you have 5 if statements in 
a row, and every one of them gets hit at one time or another, but in some 
cases the execution path goes through 3 of them or 2 of them or whatever, but 
not every single combination is tested? That would be difficult to do given that 
there's a good chance that certain combinations of branches should never 
actually occur. It would also complicate the implementation of -cov 
considerably I would think.

>   * Coverage testing of defensive code (7.2)
>     - Some branches are always (or never) executed, which can mean 100%
> branch coverage is not possible unless you remove the "defensive" code.
>     - SQLite solves this with the ALWAYS() and NEVER() macros
>     - This would only become relevant if -cov was improved

It already happens. If you use assert(0) in your code anywhere (prime examples 
of that being in the default case of a switch statement which should never 
happen or in a catch block which should never be hit but is required to make 
the function nothrow because a function in the try block theoretically _could_ 
throw but never actually will with the arguments that it's given in that 
case). std.datetime has very high code coverage (100% aside from the assert(0) 
lines, I believe), but the assert(0) lines are naturally never hit (it would 
be a bug if they were), so -cov does not actually give it 100% (it gets 
something more like 98%).

- Jonathan M Davis