Parallel execution of unittests

[w0rp via Digitalmars-d]

On Thursday, 1 May 2014 at 17:04:53 UTC, Xavier Bigand wrote:
> Le 01/05/2014 16:01, Atila Neves a écrit :
>> On Thursday, 1 May 2014 at 11:44:12 UTC, w0rp wrote:
>>> On Thursday, 1 May 2014 at 11:05:55 UTC, Jacob Carlborg wrote:
>>>> On 2014-04-30 23:35, Andrei Alexandrescu wrote:
>>>>
>>>>> Agreed. I think we should look into parallelizing all 
>>>>> unittests. --
>>>>> Andrei
>>>>
>>>> I recommend running the tests in random order as well.
>>>
>>> This is a bad idea. Tests could fail only some of the time. 
>>> Even if
>>> bugs are missed, I would prefer it if tests did exactly the 
>>> same thing
>>> every time.
>>
>> They _should_ do exactly the same thing every time. Which is 
>> why running
>> in threads or at random is a great way to enforce that.
>>
>> Atila
> +1

Tests shouldn't be run in a random order all of the time, perhaps 
once in a while, manually. Having continuous integration randomly 
report build failures is crap. Either you should always see a 
build failure, or you shouldn't see it. You can only test things 
which are deterministic, at least as far as what you observe. 
Running tests in a random order should be something you do 
manually, only when you have some ability to figure out why the 
tests just failed.