Garbage Collector and Foreign Threads

[Frits van Bommel]

Daniel Keep wrote:
> 
> will75g wrote:
>> If I understand correctly how the D garbage collector works, in order to
>> work properly it needs to stop every thread before performing collection.
>>
>> What happens if one or more of the application threads have been created
>> outside the control of the D runtime? For example imagine a C/C++
>> application (of which I don't even have the source code) loading a
>> plug-in written by me in D... Since the D runtime knows only the threads
>> created in D, the threads created by the C/C++ application could be
>> still running and calling D code while the D runtime is performing a
>> collection.
>>
>> So far the only solution I can think of is suspending the GC while my D
>> code is called from the C/C++ application.
>> My fear is that such a solution would lead to a lot of problems. For
>> example if I have to allocate and release frequently memory, there will
>> be a point where the D heap is exhausted, since the memory that I
>> release won't be collected until the GC is enabled again. At that point
>> every memory allocation will generate a
>> std.outofmemory.OutOfMemoryException, but there's nothing I can do to
>> give the collector more memory.
>>
>> Any idea?
> 
> In so far as I'm aware, there shouldn't be any problems with leaving the
> GC enabled when your application is used as a plugin to a C app: the C
> app shouldn't have any GC'ed memory (it really, really shouldn't), thus
> its threads aren't a problem.
> 
> So long as the GC knows about all the threads that *do* use GC'ed
> memory, you're fine.

In fact, as long as the GC knows of at least one pointer to any object 
the application can still reference, you should be fine. So as long as 
any GC'ed objects referenced in other threads are also referenced from a 
D thread or from global variables the GC knows about you can even 
reference GC'ed objects in non-D threads.
(At least for the current implementation, this might blow up with a 
moving collector)

> Incidentally, the GC only (currently) runs if you try to 'new'
> something, and you're out of memory.  So unless the host application is

Or you concatenate arrays (~ or ~=) or increase the .length of an array 
manually...
Or you call any Phobos routine that does any of those things...

> calling into your plugin from multiple threads simultaneously, I don't
> think you need to worry about the GC running and then having your plugin
> called in the middle of that.


Also, normally C/C++ applications keep track of when objects can be 
freed in some way. As long as it then notifies your D library it no 
longer has any references to it (instead of trying to free()/delete them 
itself) you should be able to keep a reference the GC is aware of until 
then (in a global AA, for instance) so it doesn't get prematurely 
deleted and then remove that reference. If no D code has any references 
to it, it should then be picked up by the next GC run.
Not that I've ever done this, but theoretically it should work...