Ideas for runtime loading of shared libraries.

[Sean Kelly]

The trick seems to be mapping in TLS (on OSX anyway) and running static crore at the right time. Are there other issues as well?

Sent from my iPhone

On Jan 3, 2012, at 8:53 AM, "Martin Nowak" <dawg at dawgfoto.de> wrote:

> On Tue, 03 Jan 2012 08:20:38 +0100, Jacob Carlborg <doob at me.com> wrote:
> 
>> On 2012-01-02 21:57, Martin Nowak wrote:
>>> On Mon, 02 Jan 2012 20:38:50 +0100, Jacob Carlborg <doob at me.com> wrote:
>>> 
>>>> On 2012-01-02 20:20, Martin Nowak wrote:
>>>>> I think that I'll defer the support for runtime loading of shared
>>>>> library (plugins)
>>>>> in favor of getting linked shared library support done now.
>>>>> There are several issues that require more thoughts.
>>>>> 
>>>>> - Per-thread initialization of modules is somewhat tricky.
>>>>> Doing it in Runtime.loadLibrary requires knowledge of shared library
>>>>> dependencies
>>>>> because different threads might share dependencies but this is not
>>>>> provided by libc/libdl.
>>>>> 
>>>>> - Libraries might not be unloaded as long as GC collected class
>>>>> instances still exist because
>>>>> finalization fails otherwise.
>>>>> 
>>>>> - Getting symbols through mangled names is difficult/unstable.
>>>>> 
>>>>> - D libraries used by a C library should provide proper runtime
>>>>> initialization
>>>>> even if the C library is used by a D application.
>>>>> 
>>>>> Any ideas or use-cases for plugins are welcome.
>>>>> 
>>>>> martin
>>>> 
>>>> 
>>>> - Initializing module infos
>>>> - Initializing exception handling tables
>>>> - Running module constructors
>>>> - Initializing TLS
>>>> 
>>>> Then also unload all this when the library is unloaded.
>>>> 
>>> It seems that libraries can't be unloaded deterministically,
>>> because GC finalization still references them.
>>> 
>>>> On Mac OS X, can't "_dyld_register_func_for_add_image" be used? Then
>>>> it will work, hopefully, transparently for the user. D libraries used
>>>> by C wouldn't need any different handling. Because they will be linked
>>>> with druntime it can initializing everything with the help of
>>>> "_dyld_register_func_for_add_image".
>>>> 
>>> 
>>> That was the approach I took and it is partly a dead-end.
>>> 
>>> I have a mechanism similar to _dyld_register_func_for_add_image
>>> but runtime loaders have no notion of per-thread initialization,
>>> i.e. when two threads load the same library only the first one will
>>> actually cause the image to be loaded.
>>> This implies that the second thread would need to check all
>>> dependencies of the loaded library to do the initialization.
>>> I've written something along this line but it requires to
>>> exploit/rewrite part of the runtime linker.
>>> Using dlmopen on linux would be a terrible inefficient hack
>>> around this issue, it allows to load libraries multiple times.
>> 
>> I'm not quite sure I understand. Most of the things that should be done, initializing module infos and so on, should only be done once.
>> 
> Yes most, but not all.
> The core issue here is that C++'s __thread doesn't allow dynamic initializers,
> thus there is no infrastructure to do such things. And really a clean approach
> would be to extend libc/ld.so.