Dynamic loading, D all the way (dmd 64bit 2.060/Ubuntu 64bit 12.04/x86_64)

[Carl Sturtivant]

>
> I'm not sure I'm following what you exactly have done here but 
> in general this is what needs to be done to make dynamic 
> libraries properly work in D :
>
> * Initialize module infos (module constructors and similar)
> * Add TLS variables
> * Add exception handling tables
> * Add GC roots
>
> The above four things need to be extracted from the loaded 
> dynamic library and it gets loaded and preferably remove them 
> as well when the dynamic library gets unloaded. Currently this 
> is only extracted from the running executable. This is platform 
> dependent but usually it's extracted using bracketed sections 
> via extern C variables.

OK, good to know. Any further hints about these, or where I can 
look?

What I've done is use the C dynamic loading library (header 
dlfcn.h) to manually load a shared object written in D, dload.d, 
from a D program (main.d), which then successfully calls 
functions in dload.d that are not defined extern(C).

I am attempting a do-it-yourself dynamic loading in D, where I 
explicitly do all the administration manually to make it work, 
rather than rely upon D to do it automatically. Hence my use of 
the C dynamic loading library, which knows nothing of additional 
work D must do.

Reason for this approach: the newsgroups indicate that dynamic 
loading in D does not work as yet.

What's novel is that I explicitly excluded from the shared object 
anything but code generated directly from dload.d:

dmd -c dload.d -m64 -fPIC
ld dload.o -shared -o libdload.so -m elf_x86_64 -E

And I included the whole of libphobos.a in the build of the main 
program:

dmd -c main.d -m64
gcc main.o -o main -m64 -Wl,-E -ldl -Wl,--whole-archive -lphobos2 
\
-Wl,--no-whole-archive -lcurl -lpthread -lm -lrt

This tactic is in the hope that all parts of D used in the shared 
library will find their linkage in the main program when it's 
dynamically loaded. (And the -E option passed to the linker in 
both cases is to expose all symbols for exactly this purpose.)

Reason for this approach: the newsgroups indicate that D 
runtime/phobos (all currently in libphobos.a it seems) does not 
initialize properly in a shared library, so I ensure that it's 
not present at all, and endeavor to have the shared object 
implicitly use the properly working D runtime/phobos in the main 
executable.

[And besides, at 64 bits all code in shared libraries apparently 
must be position independent, so even if I wanted to link parts 
of libphobos.a into the shared library, I couldn't without 
recompiling libphobos.a with the -fPIC option!]

This bare-bones-in-the-shared-library approach has worked well in 
my toy example. The only thing that apparently doesn't work is if 
an exception is thrown from the static initializer 'static 
this()' in the shared object. If an exception results from a call 
chain initiated by the main program even if thrown from a 
function in the shared object, all is well it seems.

Incidentally, I fibbed slightly about the ld command used to link 
the shared object. It also contains a trailing

   -init=$(shell staticCtor dload.o)

which enables the linker to bind in execution of the static 
initializer 'static this()' so that it runs automatically when 
the shared library is loaded. staticCtor is a script that 
analyzes dload.o to find its name, which is mangled of course.

If anyone can give me any more specific information about what 
else I can make happen manually to complete effective linkage at 
the D level I'd be grateful. I'm not stopping the investigation 
here!