Initializing D runtime and executing module and TLS ctors for D libraries

[Ali Çehreli]

On 1/24/21 2:28 AM, IGotD- wrote:

 > Any threads started by druntime has proper initialization of course. Any
 > thread started by any module written in another language will not do D
 > the thread initialization.

And that of course has been what I've been trying to deal with. Bugs in 
the uses of thread_attachThis and thread_detachThis, and most 
importantly, not having a guaranteed opportunity to call 
thread_detachThis (think a foreign thread dies on its own without 
calling us and the runtime crashes attempting to stop a nonexisting 
thread during a GC cycle) finally made me realize that D shared library 
functions cannot be called on foreign threads. At least not today... Or, 
they can only be used under unusual conventions like the rule below.

So, that's the golden rule: If you want to call functions of a D shared 
library (I guess static library as well) you must create your thread by 
our library's create_thread() function and join that thread by our 
library's join_thread() function. Works like a charm!

Luckily, it is trivial to determine whether we are being called on a 
foreign thread or a D thread through a module scoped bool variable...

 > Since std::function cannot be
 > used in a generic interface I actually use something like this,
 > http://blog.coldflake.com/posts/C++-delegates-on-steroids/.

If I understand that article correctly, and by pure coincidence, the 
very shared libraries that are the subject of this discussion, which I 
load at run time, happen to register themselves by providing function 
pointers. Like in the article, those function pointers are of template 
instances, each of which know exactly what to do for their particular 
types but the registry keeps opaque functions. Pseudo code:

```
// Shared library:

struct A {
   // ...
}

struct B {
   // ...
}

shared static this() {
   register("some key",
            &serializer!A,    // <-- Takes e.g. void* but knows about A
            &deserializer!B); // ditto for B
}
```

And one of my issues has been that module constructors not being called 
when the library is loaded as a dependency of a C++ library, which is 
loaded by a Python module, which is imported by another Python module. :)

As I said earlier, I solved that issue by parsing and persisting the 
output of 'nm mylib.so' to identify the module ctor and to call it after 
dlsym'ing. Pretty hacky but works...

Getting back to my main issue: I am about to write a mixin template 
where any library's interface .d file will do the following and get the 
create_thread and join_thread functions automatically:

// mylib's extern(C) functions:

// This one provides mylib_create_thread() and mylib_join_thread():
mixin LibAPI!"mylib"();

// Other extern(C) functions of the library:
extern(C)
nothrow
int foo(int) {
   // ...
}

The .h file must still be maintained by hand.

Ali