Parallel foreach gets executed one after another

[Thorsten Sommer via Digitalmars-d]

Dear Community.

Thank you for helping me so much in the last thread concerning 
our AI library 
(https://forum.dlang.org/thread/rrpmgzqqtkqgeicjdgps@forum.dlang.org). Thanks to you, we can now debug our AI library with GDB. My suspicion was confirmed: At least one of our parallel foreach loops will only be processed item by item instead of parallel.

It is the loop that contains the largest workload: An external 
process is started for each agent. I log the PID, so I see that 
the processes are started one after the other.

Our foreach loops are constructed as:

auto taskPool = new TaskPool();
foreach (n, individual; taskPool.parallel(individuals, 
params.workUnitSize4ParallelExecution)) {
     [...]
     pipes = pipeProcess(args, Redirect.stdout | Redirect.stdin);
     [...]
}

taskPool.finish(true);

The looped individuals are sourced by such a statement:
return this.individuals.filter!( [...] ).array;

Currently, for debugging, params.workUnitSize4ParallelExecution 
is set to 100 and individuals contain approx. 10 agents. 
Therefore, we would expect 10 external processes to be started 
simultaneously. Each process runs approx. 4 seconds. Accordingly, 
the processing should take about 4 seconds instead of 40 seconds.

Do you have any ideas what might be the reason for this? If you 
don't know a solution, then maybe approaches to further 
investigation?


Best regards
Thorsten