Debugging a Memory Leak

[Steven Schveighoffer via Digitalmars-d-learn]

On 11/17/14 6:12 PM, Maxime Chevalier-Boisvert wrote:
> There seems to be a memory leak in the Higgs compiler. This problem
> shows up when running our test suite (`make test` command).
>
> A new VM object is created for each unittest block, e.g.:
> https://github.com/maximecb/Higgs/blob/master/source/runtime/tests.d#L201
>
> These VM objects are unfortunately *never freed*. Not until the whole
> series of tests is run and the process terminates. The VM objects keep
> references to many other objects, and so the process keeps using more
> and more memory, up to over 2GB.
>
> The VM allocates it's own JS data heap that it manages itself, i.e.:
> https://github.com/maximecb/Higgs/blob/master/source/runtime/gc.d#L186
>
> This memory is clearly marked as NO_SCAN, and so references to the VM in
> there should presumably not be counted. There is also executable memory
> I allocate with mmap, but this should also be ignored by the D GC in
> principle (I do not mark executable code as roots):
> https://github.com/maximecb/Higgs/blob/master/source/jit/codeblock.d#L129
>
> I don't know where the problem lies. There could be false pointers, but
> I'm on a 64-bit system, which should presumably make this less likely. I
> wish there was a way to ask the D runtime "can you tell me what is
> pointing to this object?", but the situation is more complex because
> many objects in my system refer to the VM object, there is a complicated
> graph of references. If anything points into that graph, the whole thing
> stays "live".

Hm... such a function could be created. However, it would be tricky to 
make work.

First, you would need a way to store the pointer without having it 
actually point at the data. Clearly, if you pass the pointer to the 
function, it's going to be on the stack, so that would then refer to it. 
You have to somehow obfuscate it the whole time.

Second, you may be given "memory x is pointing at your target", but what 
does memory x actually mean? That isn't something the GC can deal with. 
Perhaps when precise scanning is included (and I think we are close on 
that), you will have at least some type info.

> Help or advice on solving this problem is welcome.

GC problems are *nasty*. My advice is to run the simplest program you 
can think of that still exhibits the problem, and then put in printf 
debugging everywhere to see where it breaks down.

Not sure if this is useful.

-Steve