Parallel execution of unittests

Bruno Medeiros via Digitalmars-d digitalmars-d at puremagic.com
Tue May 6 09:26:27 PDT 2014


On 01/05/2014 18:12, Steven Schveighoffer wrote:
> On Thu, 01 May 2014 10:01:19 -0400, Atila Neves <atila.neves at gmail.com>
> wrote:
>
>> On Thursday, 1 May 2014 at 11:44:12 UTC, w0rp wrote:
>>> On Thursday, 1 May 2014 at 11:05:55 UTC, Jacob Carlborg wrote:
>>>> On 2014-04-30 23:35, Andrei Alexandrescu wrote:
>>>>
>>>>> Agreed. I think we should look into parallelizing all unittests. --
>>>>> Andrei
>>>>
>>>> I recommend running the tests in random order as well.
>>>
>>> This is a bad idea. Tests could fail only some of the time. Even if
>>> bugs are missed, I would prefer it if tests did exactly the same
>>> thing every time.
>>
>> They _should_ do exactly the same thing every time. Which is why
>> running in threads or at random is a great way to enforce that.
>
> But not a great way to debug it.
>
> If your test failure depends on ordering, then the next run will be
> random too.
>

I agree with Steven here.

Actually, even if the failure does *not* depend on ordering, it can 
still be useful to run the tests in order when debugging:
If there is a bug in a low level component, that will likely trigger a 
failure in the tests for that low level component, but also the tests 
for higher-level components (the components that use the low level 
component).
As such, when debugging, you would want to run the low-level test first 
since it will likely be easier to debug the issue there, than with the 
higher-level test.

Sure, one could say that the solution to this should be mocking the 
low-level component in the high-level test, but mocking is not always 
desirable or practical. I can provide some concrete examples.

-- 
Bruno Medeiros
https://twitter.com/brunodomedeiros


More information about the Digitalmars-d mailing list