Unit tests in D

[bearophile]

Don:

>I think the majority of the items in your list can already be done fairly well (or easily enough by a library), except for giving names to unit tests.<

Let's see the points:

1) The following syntax is compatibile with D2 to implement the two parts of point 1:
assert(throws!(Ex1, Ex2)(foo(10)));
static assert(!__traits(compiles, foo(10)));
This works well enough, but the second syntax is both common enough in my code and ugly enough (ugly also means bug-prone) that some syntax improvement can be good.
Future syntax like this can improve the situation a bit:
static assert(!meta.compiles(foo(10)));

3) Currently unittest errors report only the module name and line number. If unittests get a name, but such name isn't shown up in the unittest errors, the IDE has to parse the source code to know what's the name of the failed unittest (if the IDE can't know this name, then there is little point in giving names to unittests in the first place). This is bad, so I think this tiny feature has to be built-in.

4) I think that currently the JSON ignores unittests (http://dsource.org/projects/dmd/changeset/453 ). For the IDE it's useful to know what unittests are present in a module, for example to compute the percentage of failed unittests, etc. If the JSON doesn't list them, then the IDE has to parse the code to find them by itself. This is doable, but the JSON was invented to avoid this. So I think this little feature has to be built-in.

5) This is not so important, the compiler can just ignore the unittest ddocs. The IDE can find them (knowing the starting line of the unittests) and use them as necessary to answer the user questions. Putting such unittest ddocs in the JSON can simplify the IDE a little, but this is not a vital thing. I think this is simple enough to implement that it's better for this to be built-in.

6) All serious unit test systems have several ways to disable unittests, this is a very common need. And a way to group them is often a need. D language has version(){} statements, and debug(){} too, they can be used to enable and disable unittests with no need of extra syntax support. With enough code to juggle defined and undefined symbols you can even support groups of unittests. But this is true for the debug() too, you can emulate the purpose of debug() given version() or version() given debug(). Some syntax support can turn something ad hoc and hairy in something simpler and more elegant. So some support for disabling unittests and grouping them can be useful.


The three half-baked things need more thinking, I invite people to give opinions and suggestions.

B) I don't know if this is necessary.

C) This is nice, but I don't know if this is worth the complexity to implement it. Opinions welcome.



>One thing which you've not mentioned is in unittests for interfaces (and derived classes). I would like to be able to check that *all* implementations of an interface satisfy a sequence of tests.<

A "sequence of tests" can mean a group of tests, see point 6.

So what you ask for can be seen as related to the point C, to associate a group of tests to a interface (and all its implementation tree). A possible way to implement this is with an attribute:

@dotest(group1) interface Foo {...} // association interface <=> unittest group

@dotest(test1) int bar(int x) {...} // association function <=> unittest

I am not sure if this is a good way to implement this idea.

Bye,
bearophile