Druntime and non-D threads

[Ali Çehreli]

I'm trying to use D as a library to be called from a non-D environment 
e.g. Java runtime. If I'm not mistaken, it's quite difficult and perhaps 
impossible to use GC in such a scenario. It works as long as attached 
threads don't go away either by themselves or by thread_detachThis.

My setup is Linux (Ubuntu-based), dmd 2.077.1, 64-bit build. D is used 
in a shared library that is called by non-D threads. (Tested with C and 
Java.)

1) The following newsgroup topic is about calling thread_attachThis() 
for threads created outside of D:

   http://forum.dlang.org/post/ounui4$171a$1@digitalmars.com

As suggested in that thread, I think I have to call thread_detachThis 
but I'm not sure when that can be safely done. One idea was to attach 
and detach in every api function something to the effect of

extern(C) my_api_func() {
     thread_attachThis();
     scope(exit) thread_detachThis();

     // Do work, potentially producing garbage...
}

Does that make sense? Wouldn't garbage produced by that thread leaked 
after detaching? However, failing to detach would be bad as well as the 
calling thread can terminate without our knowledge. (More on that below.)

2) Obviously, Runtime.initialize() must be called for Druntime to work 
at all. Question: Is the thread that calls Runtime.initialize() special 
compared to the other threads? Can this thread disappear and the 
Druntime still work?

3) An attached non-D thread can exit without any notice (gracefully or 
otherwise) while it's still attached to D's GC, causing segmentation 
faults or deadlock.

I failed to find a way for Druntime to be resilient when such threads 
disappear. For example, the registered cleanup handler in thread.d is 
called only for cancelled threads, not the ones that exit simply by 
returning from their thread functions. (This is according to cleanup 
handler spec.)

4) Druntime uses pthread_kill to signal threads to suspend (and resume) 
threads. However, successful return of this function does not mean that 
the thread will respond to that signal. So, we have a couple of bugs in 
Druntime as the number of sem_wait() calls we make depends on the 
unreliable return value of pthread_kill. Perhaps that's the reason for 
bugs like the following:

   https://issues.dlang.org/show_bug.cgi?id=15939

I don't see a way out of this POSIX limitation. (pthread_key_create may 
help as a "thread destructor" but I haven't played with it yet. thread.d 
beat me up pretty bad for more than two days; I'm too tired to do 
anything else right now. :) )

5) We depend on SIGUSR1 (and SIGUSR2, which may not be necessary but 
it's a different topic) to suspend non-D threads. Does that work with 
all threads? What if the calling framework has other uses for those 
signals? Would we be interfering with them?

So, what are the rules of using D as a library for a non-D framework? I 
have the following so far but I'm not sure on all points:

- SURE: One thread must make a call to Runtime.initialize()

- SURE: Every D api call must call thread_attachThis

- SURE: Attached threads must *not* terminate gracefully, due to error, 
or by cancellation. (As there is no way of guaranteeing this in POSIX, I 
think using D as a library in a framework is best-effort at best.)

- NOT SURE: thread_detachThis must *not* be called as the thread may 
have uncollected garbage.

- NOT SURE: SIGUSR1 and SIGUSR2 should be available.

Ali