Parallel execution of unittests

[Jacob Carlborg via Digitalmars-d]

On 2014-04-30 17:43, Andrei Alexandrescu wrote:
> Hello,
>
>
> A coworker mentioned the idea that unittests could be run in parallel
> (using e.g. a thread pool). I've rigged things to run in parallel
> unittests across modules, and that works well. However, this is too
> coarse-grained - it would be great if each unittest could be pooled
> across the thread pool. That's more difficult to implement.

Can't we just collect all unit tests with __traits(getUnitTests) and put 
them through std.parallelism:

foreach (unitTest ; unitTests.parallel)
     unitTest();

> This brings up the issue of naming unittests. It's becoming increasingly
> obvious that anonymous unittests don't quite scale - coworkers are
> increasingly talking about "the unittest at line 2035 is failing" and
> such. With unittests executing in multiple threads and issuing e.g.
> logging output, this is only likely to become more exacerbated. We've
> resisted named unittests but I think there's enough evidence to make the
> change.

Named unit tests are already possible with the help of UDA's:

@name("foo bar") unittest
{
     assert(true);
}

I've tried several times here, in reviews, to get people to add some 
description to the unit tests. But so far no one has agreed.

I'm using something quite similar to RSpec from the Ruby world:

describe! "toMsec" in {
     it! "returns the time in milliseconds" in {
         assert(true);
     }
}

This uses the old syntax, with UDA's it becomes something like this:

@describe("toMsec")
{
     @it("returns the time in milliseconds") unittest
     {
         assert(true);
     }
}

> Last but not least, virtually nobody I know runs unittests and then
> main. This is quickly becoming an idiom:
>
> version(unittest) void main() {}
> else void main()
> {
>     ...
> }

Or "dmd -unittest -main -run foo.d"

> I think it's time to change that. We could do it the
> non-backward-compatible way by redefining -unittest to instruct the
> compiler to not run main. Or we could define another flag such as
> -unittest-only and then deprecate the existing one.

Fine by me, I don't like that "main" is run after the unit tests.

> Thoughts? Would anyone want to work on such stuff?

Are you thinking of built-in support or an external library?

-- 
/Jacob Carlborg