Improving unit tests

[Sean Kelly]

Christopher Wright wrote:
> Janderson wrote:
> 
>> 3) If you chose "more extensible unit tests" what language features 
>> would be needed to make this happen.
> 
> 1. Named unittests.
> There are a few reasons for this:
>     - Your code should be self-documenting. unittest {} tells you nothing.
>     - If a test fails, you want to know which test it is. A backtrace 
> library helps more than a unittest name, I admit, but both are useful.
>     - Naming for unittests integrates better with continuous integration 
> servers like CruiseControl and CCNET -- these expect unittests to have 
> fully qualified names.

I agree that this would be nice.  Perhaps unittest("name") for the syntax?

> 2. A pluggable unittest runner.
> With this, any functionality that unittests lack can be provided. The 
> unittest runner has to be in the runtime, of course.

There's one in druntime.

> 3. Each test should be its own function.
> Currently, all unittest blocks in any module are conglomerated into one 
> function, ModuleInfo.unitTest. ModuleInfo should have a list of 
> name/function pairs. This is required for named unittests, and for 
> reporting errors on a per-test basis.

This would be nice as well.  Recovering from unit tests currently skips 
all tests in a module after the test that failed, which isn't terribly 
desirable.

> 4. The unittest runner must be able to have runtime arguments.
> At work, we use test-driven development. We have about 5700 classes, 
> half of which are test classes. That's 7,200 tests currently. They take 
> ten minutes to run. You can't do TDD with ten minute lag for running tests.
> 
> Command line arguments are the easiest way of changing options at 
> runtime. The alternatives are some sort of interactive session with 
> stdio, an options file, or recompiling (which isn't runtime). None of 
> these is particularly good.

Hm... perhaps if argc/argv were passed to the unit test runner from the 
runtime?  This would leave the choice of switches up to the user.


Sean