Parallel execution of unittests

[Andrei Alexandrescu via Digitalmars-d]

On 5/1/14, 2:28 PM, Jason Spencer wrote:
> On Thursday, 1 May 2014 at 17:57:05 UTC, Andrei Alexandrescu wrote:
>> Well how complicated can we make it all? -- Andrei
>
> As simple as possible, but no simpler :)
>
> I've seen you favor this or that feature because it would make unit
> testing easier and more accessible, and eschew features that would cause
> folks to not bother.  In truth, we could leave it how it is.  But I
> surmise you started this thread to improve the feature and encourage
> more use of unit test.  So we're looking for the sweet spot.
>
> I don't think it's important to support the sharing of state between
> unit tests.  But I do see value in being able to influence the order of
> test execution, largely for debugging reasons.  It's important for a
> module's tests to be able to depend on other modules--otherwise,
> unittest is not very enticing.  If it does and there's a failure, it's
> hugely helpful to know the failure is caused by the unit-under-test, and
> not the dependency(s).  The common way to do that is to run the tests in
> reverse order of dependency--i.e. levelize the design and test from the
> bottom up.  See "Large Scale C++ SW Design", Lakos, Chp. 3-4.
>
> I imagine there are other niche reasons for order, but for me, this is
> the driving reason.  So it seems a nice middle ground.
>
> If order is important, it might be a workable approach to run unittest
> in the reverse module dependency order by default.  A careful programmer
> could arrange those classes/functions in modules to take advantage of
> that order if it were important. Seems like we'd have the dependency
> information--building and traversing a tree shouldn't be that tough....
> To preserve it, you'd only be able to parallelize the UTs within a
> module at a time (unless there's a different flag or something.)
>
> But it seems the key question is whether order can EVER be important for
> any reason.  I for one would be willing to give up parallelization to
> get levelized tests.  What are you seeing on your project?  How do you
> allow tests to have dependencies and avoid order issues?  Why is
> parallelization more important than that?

I'll be blunt. What you say is technically sound (which is probably why 
you believe it is notable) but seems to me an unnecessarily complex 
engineering contraption that in all likelihood has more misuses than 
good uses. I fully understand you may think I'm a complete chowderhead 
for saying this; in the past I've been in your place and others have 
been in mine, and it took me years to appreciate both positions. -- Andrei