Parallel execution of unittests

[Jonathan M Davis via Digitalmars-d]

On Wed, 30 Apr 2014 23:56:53 -0700
Andrei Alexandrescu via Digitalmars-d <digitalmars-d at puremagic.com>
wrote:
> I don't think undefined behavior is at stake here, and I find the
> simile invalid. Thread isolation is a done deal in D and we may as
> well take advantage of it. Worse that could happen is that a unittest
> sets a global and surprisingly the next one doesn't "see" it.
> 
> At any rate I think it's pointless to insist on limiting parallel 
> running to pure - let me just say I understood the point (thanks) so 
> there is no need to restate it, and that I think it doesn't take us a 
> good place.

I'm only arguing for using pure on the grounds that it _guarantees_
that the unittest block is safely parallelizable. If we decide that that
guarantee isn't necessary, then we decide that it isn't necessary,
though I definitely worry that not having that guarantee will be
problematic. I do agree though that D's thread-local by default helps
quite a bit in ensuring that most tests will be runnable in parallel.
However, if we went with purity to indicate parallelizability, I could
easily see doing it implicitly based on purity and allowing for a UDA
or somesuch which marked a unittest block as "trusted pure" so that it
could be run in parallel. So, I don't think that going with pure would
necessarily be too restrictive. It just would require that the
programmer do some extra work to be able to treat a unittest block as
safely parallelizable when the compiler couldn't guarantee that it was.

Ultimately, my biggest concern here is that it be possible to guarantee
that a unittest block is not run in parallel with any other unittest
block if that particular unittest requires it for any reason, and some
folks seem to be arguing that such tests are always invalid, and I
want to make sure that we don't ever consider that to be the case for
unittest blocks in D. If we do parallel by default and allow for some
kind of markup to make a unittest block serial, then that can work. I
fully expect that switching to parallel by default would break a number
of tests, which I do think is a problem (particularly since a number
of those tests will be completely valid), but it could also be an
acceptable one - especially if for the most part, the code that it
breaks is badly written code. Regardless, we will need to make sure that
we message the change clearly in order to ensure that a minimal number
of people end up with random test failures due to the change.

On a side note, regardless of whether we want to use purity to infer
paralellizability, I think that it's very cool that we have the
capability to do so if we so choose, whereas most other languages
have no way of even coming close to being able to tell whether a
function can be safely parallelized or not. The combination of
attributes such as pure and compile-time inference is very cool indeed.

- Jonathan M Davis