problem with parallel foreach

[Gerald Jansen via Digitalmars-d-learn]

On Tuesday, 12 May 2015 at 20:58:16 UTC, Vladimir Panteleev wrote:
> On Tuesday, 12 May 2015 at 18:14:56 UTC, Gerald Jansen wrote:
>> On Tuesday, 12 May 2015 at 16:35:23 UTC, Rikki Cattermole 
>> wrote:
>>> On 13/05/2015 4:20 a.m., Gerald Jansen wrote:
>>>> At the risk of great embarassment ... here's my program:
>>>> http://dekoppel.eu/tmp/pedupg.d
>>>
>>> Would it be possible to give us some example data?
>>> I might give it a go to try rewriting it tomorrow.
>>
>> http://dekoppel.eu/tmp/pedupgLarge.tar.gz (89 Mb)
>>
>> Contains two largish datasets in a directory structure 
>> expected by the program.
>
> Profiling shows that your program spends most of the time 
> reading the data.
>
> I see a considerable speed boost with the following one-line 
> patch (plus imports):
>
> - foreach (line; File(pednum, "r").byLine()) {
> + foreach (line; (cast(string)read(pednum)).split('\n')) {

Nice, thanks. Making that replacement in three points in the 
program resulted in roughly a 30% speedup at the cost of about 
30% more memory in this specific case. Unfortunately it didn't 
help with the performance deteroration problem with parallel 
foreach.