problem with parallel foreach

[weaselcat via Digitalmars-d-learn]

On Wednesday, 13 May 2015 at 09:01:05 UTC, Gerald Jansen wrote:
> On Wednesday, 13 May 2015 at 03:19:17 UTC, thedeemon wrote:
>> In case of Python's parallel.Pool() separate processes do the 
>> work without any synchronization issues. In case of D's 
>> std.parallelism it's just threads inside one process and they 
>> do fight for some locks, thus this result.
>
> Okay, so to do something equivalent I would need to use 
> std.process. My next question is how to pass the common data to 
> the sub-processes. In the Python approach I guess this is 
> automatically looked after by pickling serialization. Is there 
> something similar in D? Alternatively, would the use of 
> std.mmfile to temporarily store the common data be a reasonable 
> approach?

Assuming you're on a POSIX compliant platform, you would just 
take advantage of fork()'s shared memory model and pipes - i.e, 
read the data, then fork in a loop to process it, then use pipes 
to communicate. It ran about 3x faster for me by doing this, and 
obviously scales with the workloads you have(the provided data 
only seems to have 2.) If you could provide a larger dataset and 
the python implementation, that would be great.

I'm actually surprised and disappointed that there isn't a 
fork()-backend to std.process OR std.parallel. You have to use 
stdc