problem with parallel foreach

[Gerald Jansen via Digitalmars-d-learn]

On Tuesday, 12 May 2015 at 16:46:42 UTC, thedeemon wrote:
> On Tuesday, 12 May 2015 at 14:59:38 UTC, Gerald Jansen wrote:
>
>> The output of /usr/bin/time is as follows:
>>
>> Lang Jobs    User  System  Elapsed %CPU
>> Py      2   79.24    2.16  0:48.90  166
>> D       2   19.41   10.14  0:17.96  164
>>
>> Py     30 1255.17   58.38  2:39.54  823 * Pool(12)
>> D      30  421.61 4565.97  6:33.73 1241
>
> The fact that most of the time is spent in System department is 
> quite important. I suspect there are too many system calls from 
> line-wise reading and writing the files. How many lines are 
> read and written there?

About 3.5 million lines read by main(), 0.5 to 2 million lines 
read and 3.5 million lines written by runTraits (aka runJob).

I have smaller datasets that I test on my laptop with a single 
quad-core I7 which sometimes show little increase in System time 
and other times have a marked increase, but not nearly as 
exagerated as in the large datasets on the server.

Gerald