std.parallelism amap not scaling?

safety0ff safety0ff.dev at gmail.com
Tue Oct 8 19:52:25 PDT 2013


On Tuesday, 8 October 2013 at 10:54:08 UTC, JR wrote:
> On Monday, 7 October 2013 at 21:13:53 UTC, safety0ff wrote:
>> I think I've found the culprit: Memory managment / GC, 
>> disabling the GC caused the program to eat up all my memory.
>>
>> I'll have to look into this later.
>
> From what I've gathered from 
> http://forum.dlang.org/thread/dbeliopehpsncrckdfal@forum.dlang.org, 
> your use of enum makes it copy (and allocate) those variables 
> on each access.
>
> Quoth Dmitry Olshansky in that thread (with its slightly 
> different context);
>> And the answer is - don't use ENUM with ctRegex.
>> The problem is that ctRegex returns you a pack of 
>> datastructures (=arrays).
>> Using them with enum makes it behave as if you pasted them as 
>> array literals and these do allocate each time.
>
> Merely replacing all occurences of enum with immutable seems to 
> make a world of difference. I benched your main.d a bit on this 
> laptop (also i7, so 4 real cores + HT); 
> http://dpaste.dzfl.pl/a4ecc84f4
>
> Note that inlining slows it down. I didn't verify its output, 
> but if those numbers are true then ldmd2 -O -release 
> -noboundscheck is a beast.

Thank you for responding!
I went ahead and stub'ed the gc as per: 
http://forum.dlang.org/thread/fbjeivugntvudgopyfll@forum.dlang.org
and ended coming to the same thread/conclusion.

Enum creating hidden allocations is evil. :(


More information about the Digitalmars-d-learn mailing list