Building several dynamic libraries with one shared GC

[Ali Çehreli]

On 9/12/21 7:31 AM, NonNull wrote:
 > I am making a plug-in development system for a high performance Linux
 > application that already exists and is written in C and will not be
 > modified for this purpose.

I've done something similar but in my case the main application is in D 
and the plugins are code-generated D (hand edited by non-D developers).

However, as is common in software, the requirements changed :) and we 
wanted to use it as a library as well; so, plugins became part of a main 
D library (which exposed C functions).

All initialization functions of the plugins were called automatically in 
my D test environment and all plugins were usable. The trouble started 
when the main library was being used in a foreign environment (something 
like Python loading Python loading C++ library loading our D library): 
Although the initialization function of the main library was being 
called, the 'shared static this' functions of the plugins were not being 
called.

(I tried dlopen after guessing intelligently the name of the 'shared 
static this' function (not obvious); it was not satisfactory and I don't 
remember exactly why not.)

Later I learned, this could be because merely loading a plugin might not 
be enough, and perhaps the main library might have to use a feature of 
the library as well:

   https://forum.dlang.org/post/sdb5jb$2rk3$1@digitalmars.com

If that link indeed explains the issue, I did not know about it when we 
worked around the initialization problem by running a D daemon per 
thread behind this main library.

Each thread of the library passes information to its daemon through 
shared memory and the daeman does it's thing and returns the result 
back. (The difference is, the plugins are linked to the daemon; avoiding 
the problem initialization scenario; 'shared static this' are called 
from a D environment.)

 > If several plugins are built by different third parties, each dynamic
 > library will have its own GC and copy of druntime right now.

Like the user 'frame', I don't think that's the case.

Going off topic, I started thinking about this "one daemon per library 
thread" idea: becaues I am under the impression that running multiple 
daemons does not put much stress on the system under Linux, this idea 
can automatically translate to "independent GCs for each thread". For 
example, the main library has 10 threads using 10 separate daemons on 
their backgrounds; each daemon handles its own GC needs without even 
knowing about the other daemons that are effectively working for the 
same main library. I haven't experimented with this yet.

Ali