problem with parallel foreach

[Gerald Jansen via Digitalmars-d-learn]

John Colvin's improvements to my D program seem to have resolved 
the problem.
(http://forum.dlang.org/post/ydgmzhlspvvvrbeemrqf@forum.dlang.org
and http://dpaste.dzfl.pl/114d5a6086b7).

I have rerun my tests and now the picture is a bit different (see 
tables below).

In the middle table I have used gnu parallel in combination with 
a slightly modified version of the D program which runs a single 
trait (specified in argv[1]). This approach runs the jobs as 
completely isolated processes, but at the extra cost of 
re-reading the common data for each trait. The elapsed time is 
very similar with the parallel foreach in the D program or using 
gnu parallel (for this particular program and these data run on 
this server...). I'm guessing the program is now essentially 
limited by disk I/O, so this is about as good as it gets.

So, just to wrap up:
- there is a nice speed improvement over Python program :-)
- one needs to learn a fair bit to fully benefit from D's 
potential
- thanks for all the help!

Gerald Jansen


Jobs __ time for D parallel foreach w. JC mods____
1     4.71user  0.56system 0:05.28elapsed   99%CPU
2     6.59user  0.96system 0:05.48elapsed  137%CPU
4    11.45user  1.94system 0:07.24elapsed  184%CPU
8    20.30user  5.18system 0:13.16elapsed  193%CPU
16   68.48user 13.87system 0:27.21elapsed  302%CPU
27   99.66user 18.73system 0:42.34elapsed  279%CPU

Jobs __ gnu parallel + D program for single job __
1     4.71user  0.56system 0:05.28elapsed   99%CPU as above
2     9.66user  1.28system 0:05.76elapsed  189%CPU
4    18.86user  3.85system 0:08.15elapsed  278%CPU
8    40.76user  7.53system 0:14.69elapsed  328%CPU
16  135.76user 20.68system 0:31.06elapsed  503%CPU
27  189.43user 28.26system 0:47.75elapsed  455%CPU

Jobs _____ time for python version _____________
1    45.39user  1.52system 0:46.88elapsed  100%CPU
2    77.76user  2.42system 0:47.16elapsed  170%CPU
4   141.28user  4.37system 0:48.77elapsed  298%CPU
8   280.45user  8.80system 0:56.00elapsed  516%CPU
16  926.05user 20.48system 1:31.36elapsed 1036%CPU
27 1329.09user 27.18system 2:11.79elapsed 1029%CPU