How to debug (potential) GC bugs?

[Kapps via Digitalmars-d-learn]

On Sunday, 25 September 2016 at 16:23:11 UTC, Matthias Klumpp 
wrote:
> Hello!
> I am working together with others on the D-based 
> appstream-generator[1] project, which is generating software 
> metadata for "software centers" and other package-manager 
> functionality on Linux distributions, and is used by default on 
> Debian, Ubuntu and Arch Linux.
>
> For Ubuntu, some modifications on the code were needed, and 
> apparently for them the code is currently crashing in the GC 
> collection thread: http://paste.debian.net/840490/
>
> The project is running a lot of stuff in parallel and is using 
> the GC (if the extraction is a few seconds slower due to the GC 
> being active, it doesn't matter much).
>
> We also link against a lot of 3rd-party libraries and use a big 
> amount of existing C code in the project.
>
> So, I would like to know the following things:
>
> 1) Is there any caveat when linking to C libraries and using 
> the GC in a project? So far, it seems to be working well, but 
> there have been a few cases where I was suspicious about the GC 
> actually doing something to malloc'ed stuff or C structs 
> present in the bindings.
>
> 2) How can one debug issues like the one mentioned above 
> properly? Since it seems to happen in the GC and doesn't give 
> me information on where to start searching for the issue, I am 
> a bit lost.
>
> 3) The tool seems to leak memory somewhere and OOMs pretty 
> quickly on some machines. All the stuff using C code frees 
> resources properly though, and using Valgrind on the project is 
> a pain due to large amounts of data being mmapped. I worked 
> around this a while back, but then the GC interfered with 
> Valgrind, making information less useful. Is there any 
> information on how to find memory leaks, or e.g. large structs 
> the GC cannot free because something is still having a needless 
> reference on it?
>
> Unfortunately I can't reproduce the crash from 2) myself, it 
> only seems to happen at Ubuntu (but Ubuntu is using some 
> different codepaths too).
>
> Any insights would be highly appreciated!
> Cheers,
>    Matthias
>
> [1[: https://github.com/ximion/appstream-generator

First, make sure any C threads calling D code use 
Thread.attachThis (thread_attachThis maybe?). Otherwise the GC 
will not suspend those threads during a collection which will 
cause crashes. I'd guess this is your issue.

Second, tell the GC of non-GC memory that has pointers to GC 
memory by using GC.addRange / GC.addRoot as needed. Make sure to 
remove them once the non-GC memory is deallocated as well, 
otherwise you'll get memory leaks. The GC collector is also 
conservative, not precise, so false positives are possible. If 
you're using 64 bit programs, this shouldn't be much of an issue 
though.

Finally, make sure you're not doing any GC allocations in dtors.