Parallel execution of unittests

[Jonathan M Davis via Digitalmars-d]

On Wed, 30 Apr 2014 08:59:42 -0700
Andrei Alexandrescu via Digitalmars-d <digitalmars-d at puremagic.com>
wrote:

> On 4/30/14, 8:54 AM, bearophile wrote:
> > Andrei Alexandrescu:
> >
> >> A coworker mentioned the idea that unittests could be run in
> >> parallel
> >
> > In D we have strong purity to make more safe to run code in
> > parallel:
> >
> > pure unittest {}
> 
> This doesn't follow. All unittests should be executable concurrently.
> -- Andrei
> 

In general, I agree. In reality, there are times when having state
across unit tests makes sense - especially when there's expensive setup
required for the tests. While it's not something that I generally
like to do, I know that we have instances of that where I work. Also, if
the unit tests have to deal with shared resources, they may very well be
theoretically independent but would run afoul of each other if run at
the same time - a prime example of this would be std.file, which has to
operate on the file system. I fully expect that if std.file's unit
tests were run in parallel, they would break. Unit tests involving
sockets would be another type of test which would be at high risk of
breaking, depending on what sockets they need.

Honestly, the idea of running unit tests in parallel makes me very
nervous. In general, across modules, I'd expect it to work, but there
will be occasional cases where it will break. Across the unittest
blocks in a single module, I'd be _very_ worried about breakage. There
is nothing whatsoever in the language which guarantees that running
them in parallel will work or even makes sense. All that protects us is
the convention that unit tests are usually independent of each other,
and in my experience, it's common enough that they're not independent
that I think that blindly enabling parallelization of unit tests across
a single module is definitely a bad idea.

- Jonathan M Davis