Ideas for runtime loading of shared libraries.

[Martin Nowak]

On Tue, 03 Jan 2012 08:20:38 +0100, Jacob Carlborg <doob at me.com> wrote:

> On 2012-01-02 21:57, Martin Nowak wrote:
>> On Mon, 02 Jan 2012 20:38:50 +0100, Jacob Carlborg <doob at me.com> wrote:
>>
>>> On 2012-01-02 20:20, Martin Nowak wrote:
>>>> I think that I'll defer the support for runtime loading of shared
>>>> library (plugins)
>>>> in favor of getting linked shared library support done now.
>>>> There are several issues that require more thoughts.
>>>>
>>>> - Per-thread initialization of modules is somewhat tricky.
>>>> Doing it in Runtime.loadLibrary requires knowledge of shared library
>>>> dependencies
>>>> because different threads might share dependencies but this is not
>>>> provided by libc/libdl.
>>>>
>>>> - Libraries might not be unloaded as long as GC collected class
>>>> instances still exist because
>>>> finalization fails otherwise.
>>>>
>>>> - Getting symbols through mangled names is difficult/unstable.
>>>>
>>>> - D libraries used by a C library should provide proper runtime
>>>> initialization
>>>> even if the C library is used by a D application.
>>>>
>>>> Any ideas or use-cases for plugins are welcome.
>>>>
>>>> martin
>>>
>>>
>>> - Initializing module infos
>>> - Initializing exception handling tables
>>> - Running module constructors
>>> - Initializing TLS
>>>
>>> Then also unload all this when the library is unloaded.
>>>
>> It seems that libraries can't be unloaded deterministically,
>> because GC finalization still references them.
>>
>>> On Mac OS X, can't "_dyld_register_func_for_add_image" be used? Then
>>> it will work, hopefully, transparently for the user. D libraries used
>>> by C wouldn't need any different handling. Because they will be linked
>>> with druntime it can initializing everything with the help of
>>> "_dyld_register_func_for_add_image".
>>>
>>
>> That was the approach I took and it is partly a dead-end.
>>
>> I have a mechanism similar to _dyld_register_func_for_add_image
>> but runtime loaders have no notion of per-thread initialization,
>> i.e. when two threads load the same library only the first one will
>> actually cause the image to be loaded.
>> This implies that the second thread would need to check all
>> dependencies of the loaded library to do the initialization.
>> I've written something along this line but it requires to
>> exploit/rewrite part of the runtime linker.
>> Using dlmopen on linux would be a terrible inefficient hack
>> around this issue, it allows to load libraries multiple times.
>
> I'm not quite sure I understand. Most of the things that should be done,  
> initializing module infos and so on, should only be done once.
>
Yes most, but not all.
The core issue here is that C++'s __thread doesn't allow dynamic  
initializers,
thus there is no infrastructure to do such things. And really a clean  
approach
would be to extend libc/ld.so.