Cleaning/Releasing large arrays from memory

Clinton via Digitalmars-d-learn digitalmars-d-learn at puremagic.com
Mon Jul 24 07:45:16 PDT 2017


Hi guys, I have a question on how to free large arrays in D after 
they're no longer needed.

Let's say I have this:


SomeKey[] getKeys() {
   SomeKey[] n;

   foreach(batching stuff...) {
    SomeKey newstuff = db.select!(SomeKey[])(...); // gets around 
6M of these from db

    redis.send("newItems", newStuff.length);

    n ~= newStuff;

    destroy(newStuff);
   }

   return n;
}

void main() {
   SomeKey[] myarray = getKeys();
   ulong[string] mappedValues;

   foreach(i, item; myArray) {
    mappedValues[item.name] = i;
   }

   // No longer need myarray, and we need the memory back
   destory(myarray);
   GC.collect;

   // Another memory intensive operation below
}


This seems to work for me to a point, but I notice my program 
still holds a lot in memory even after certain memory intensive 
arrays should have been collected.

Is this just a hit and miss with the GC or am I doing something 
wrong? Uses around 40% less memory forcing the GC and cleaning 
arrays but seems like a lot is left over, especially for threads 
that have already finished(I've expect a thread to free all it's 
memory). I'm not sharing anything across threads besides a config 
struct.

This is a simplified version of my app. At the moment when at 
around 9GB at it's peak instead of 19GB like before.


More information about the Digitalmars-d-learn mailing list