Parallel execution of unittests

[Dicebot via Digitalmars-d]

On Monday, 5 May 2014 at 18:58:37 UTC, Andrei Alexandrescu wrote:
> On 5/5/14, 11:47 AM, Dicebot wrote:
>> On Monday, 5 May 2014 at 18:29:40 UTC, Andrei Alexandrescu 
>> wrote:
>>> My understanding here is you're trying to make dogma out of
>>> engineering choices that may vary widely across projects and
>>> organizations. No thanks.
>>>
>>> Andrei
>>
>> I am asking to either suggest an alternative solution or to 
>> clarify why
>> you don't consider it is an important problem.
>
> "Clean /tmp/ judiciously."

This is solution for "failing test" problem. Problem I speak 
about is "figuring out why test has failed".

> The problem with your stance, i.e.:
>
>> "Unittests should do no I/O because any sort of I/O can fail 
>> because
>> of reasons you don't control from the test suite" is an 
>> appropriate
>> generalization of my statement.
>
> is that it immediately generalizes into the unreasonable:
>
> "Unittests should do no $X because any sort of $X can fail 
> because of reasons you don't control from the test suite".
>
> So that gets into machines not having any memory available, 
> with full disks etc.

It is great you have mentioned RAM here as it nicely draws a 
border-line. Being out of memory throws specific Error which is 
unlikely to be caught and clearly identifies problem. Disk I/O 
failure throws Exception which can be easily consumed somewhere 
inside tested control flow resulting in absolutely mysterious 
test failures. It is borderline of Error vs Exception - fatal 
problem incompatible with further execution and routine problem 
application is expected to handle.

> Just make sure test machines are prepared for running unittests 
> to the extent unittests are expecting them to. We're wasting 
> time trying to frame this as a problem purely related to 
> unittests alone.

Again: you don't have control of test machines for something like 
language standard library. It is not purely unittest problem, it 
is problem hard to solve staying within infrastructure of 
unittests.