Why does this simple test program leak 500MB of RAM?

FeepingCreature feepingcreature at gmail.com
Mon Jun 19 10:15:02 UTC 2023


Consider this program:

```
module test;

import core.memory : GC;
import core.sync.semaphore;
import core.thread;
import std;

enum keys = [
     "foo", "bar", "baz", "whee",
     "foo1", "foo2", "foo3", "foo4", "foo5", "foo6", "foo7", 
"foo8",
     "bar1", "bar2", "bar3", "bar4", "bar5", "bar6", "bar7", 
"bar8",
     "baz1", "baz2", "baz3", "baz4", "baz5", "baz6", "baz7", 
"baz8"];

void spamGC(Semaphore sema) {
     void recursionTest(int depth) {
         if (depth < 1000) {
             recursionTest(depth + 1);
             if (depth % 300 == 0)
                 recursionTest(depth + 1);
         }
         string[string] assocArray;
         static foreach (a; keys)
         {
             assocArray[a] = a;
         }
         // ?????
         assocArray.clear;
         assocArray = null;
     }
     recursionTest(0);
     sema.notify;
     Thread.sleep(3600.seconds);
}

void main() {
     Thread[] threads;
     auto sema = new Semaphore(0);
     enum threadCount = 100;
     threadCount.iota.each!((_) {
         auto thread = new Thread({ spamGC(sema); });
         thread.start;
         threads ~= thread;
     });
     // this clears the leak up!
     // threads.each!((thread) { thread.join; });
     threadCount.iota.each!((i) {
         sema.wait;
     });
     writefln!"Done.";
     100.iota.each!((_) {
         GC.collect;
         GC.minimize;
     });
     writefln!"Collected.";

     // Now look at residential memory for the process.
     Thread.sleep(3600.seconds);
}
```

We've had problems with excessive memory leaks in JSON decoding 
in production. So I've put together this test program.

The `spamGC` recursive call represents recursive JSON decoding.

The `Thread.sleep` call represents a thread idling in a 
threadpool.

After completion, the program sits at ~600MB residential. I 
believe, from looking at it, it should be obvious that this 
memory is **entirely dead**.

If I alloca 10KB of stack at the beginning of the recursion, the 
residential RAM drops to ~180MB. I don't think this completely 
solves the issue, but it sure is interesting.

`-gx` does nothing.


More information about the Digitalmars-d mailing list