Parallel execution of unittests

[Steven Schveighoffer via Digitalmars-d]

On Wed, 30 Apr 2014 11:43:31 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail at erdani.org> wrote:

> Hello,
>
>
> A coworker mentioned the idea that unittests could be run in parallel  
> (using e.g. a thread pool). I've rigged things to run in parallel  
> unittests across modules, and that works well. However, this is too  
> coarse-grained - it would be great if each unittest could be pooled  
> across the thread pool. That's more difficult to implement.

I am not sure, but are unit-test blocks one function each, or one function  
per module? If the latter, that would have to be changed.

> This brings up the issue of naming unittests. It's becoming increasingly  
> obvious that anonymous unittests don't quite scale - coworkers are  
> increasingly talking about "the unittest at line 2035 is failing" and  
> such. With unittests executing in multiple threads and issuing e.g.  
> logging output, this is only likely to become more exacerbated. We've  
> resisted named unittests but I think there's enough evidence to make the  
> change.

I would note this enhancement, which Walter agreed should be done at DConf  
'13 ;)

https://issues.dlang.org/show_bug.cgi?id=10023

Jacob Carlborg has tried to make this work, but the PR has not been pulled  
yet (I think it needs some updating at least, and there were some  
unresolved questions IIRC).

> Last but not least, virtually nobody I know runs unittests and then  
> main. This is quickly becoming an idiom:
>
> version(unittest) void main() {}
> else void main()
> {
>     ...
> }
>
> I think it's time to change that. We could do it the  
> non-backward-compatible way by redefining -unittest to instruct the  
> compiler to not run main. Or we could define another flag such as  
> -unittest-only and then deprecate the existing one.

The runtime can intercept this parameter. I would like a mechanism to run  
main decided at runtime.

We need no compiler modifications to effect this.

> Thoughts? Would anyone want to work on such stuff?

I can probably take a look at changing the unittests to avoid main without  
a runtime parameter. I have a good grasp on how the pre-main runtime  
works, having rewritten the module constructor algorithm a while back.

I am hesitant to run all unit tests in parallel without an opt-out  
mechanism. The above enhancement being implemented would give us some ways  
to play around, though.

-Steve