Parallel execution of unittests

[Russel Winder via Digitalmars-d]

On Wed, 2014-04-30 at 10:50 -0700, Jonathan M Davis via Digitalmars-d
wrote:
[…]
> In general, I agree. In reality, there are times when having state
> across unit tests makes sense - especially when there's expensive setup
> required for the tests. While it's not something that I generally
> like to do, I know that we have instances of that where I work. Also, if
> the unit tests have to deal with shared resources, they may very well be
> theoretically independent but would run afoul of each other if run at
> the same time - a prime example of this would be std.file, which has to
> operate on the file system. I fully expect that if std.file's unit
> tests were run in parallel, they would break. Unit tests involving
> sockets would be another type of test which would be at high risk of
> breaking, depending on what sockets they need.

Surely if there is expensive set up you are doing an integration or
system test not a unit test. In a unit test all expensive set up should
be mocked out.

> Honestly, the idea of running unit tests in parallel makes me very
> nervous. In general, across modules, I'd expect it to work, but there
> will be occasional cases where it will break. Across the unittest
> blocks in a single module, I'd be _very_ worried about breakage. There
> is nothing whatsoever in the language which guarantees that running
> them in parallel will work or even makes sense. All that protects us is
> the convention that unit tests are usually independent of each other,
> and in my experience, it's common enough that they're not independent
> that I think that blindly enabling parallelization of unit tests across
> a single module is definitely a bad idea.

All tests should be independent, therefore there should be no problem
executing all tests at the same time and/or in any order. If tests have
to be executed in a specific order then they are not separate tests and
should be merged into a single test — which likely means they are
integration or system tests not unit tests.

-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder at ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel at winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder