Shared library support: Where we are at in 2023

[Richard Andrew Cattermole (Rikki)]

Part 2 of: 
https://forum.dlang.org/post/ghsigisnciolxsiyaehu@forum.dlang.org

Rainer has been working on dmd support of DLL's, if completed 
this should fix export on Windows minus the ability to annotate 
DllImport/Internal in: https://github.com/dlang/dmd/pull/14849

So I've got a list of bugs and enhancements that'll get us 
further down the road of shared library support.

All of these feature are in my WIP export DIP, however as pretty 
much everything there could probably be implemented without the 
DIP lets start with the things I think have the greatest 
improvements in both mine and other peoples lives.

As an update to my own stuff, -betterC DLL with full D executable 
(unit testing) works even with dmd on Windows. Implementing the 
list of issues brought up here should allow anyone to repeat what 
I've done with minimal understanding of how linkers work as long 
as the appropriete symbols are annotated with export.



# Telling the compiler about symbol location

The compiler needs to know if a module is going inside of the 
binary or not (i.e. is in a dependency shared library).

1. It allows for the compiler to error if a templated symbol when 
instantiated references a non-exported symbol when the 
non-exported symbol is in another binary.
2. Allows automatic differentiation between DllImport and 
internal of symbols marked export.
3. Prevent unittests triggering the need for ModuleInfo when the 
unittest is in a module that is in an external binary.

This could be done in one of two ways, however one is the clear 
winner.

a. Use a pattern to match the full module name. This would not be 
automatic at the package/build manager level without slowing 
things down and would not be very cheap to process.
b. Use a second -I switch. This would be automatic at the 
package/build manager level and over all should be very cheap.

## Fixes:

https://issues.dlang.org/show_bug.cgi?id=22367

## Completes:

https://issues.dlang.org/show_bug.cgi?id=23850

## Removal of workaround:

ModuleInfo stubs.

https://github.com/Project-Sidero/basic_memory/blob/main/source/sidero/base/moduleinfostubs.d

## Prevents linker warnings:

https://learn.microsoft.com/en-us/cpp/error-messages/tool-errors/linker-tools-warning-lnk4217?view=msvc-170

https://learn.microsoft.com/en-us/cpp/error-messages/tool-errors/linker-tools-warning-lnk4286?view=msvc-170

This is quite an exciting possibility. Not having to mark symbols 
as DllImport/Internal will allow (in majority of cases) for these 
warnings to not occur. We will still need a way to annotate this 
information (easily solved design wise, but will leave it for 
later).

Just exporting a symbol will be enough even on Windows as long as 
the package/build manager helps out a little bit.



# Generated symbols

When the compiler generates a symbol it should always be exported 
if it has other symbols in the encapsulation unit that are 
exported or is itself exported.

Encapsulation unit in this case includes: modules, classes, 
structs, unions, template blocks.

A non-exhaustive list of symbols that should be exported:

1. TypeInfo
2. ModuleInfo
3. __init
4. __fieldDtor
5. __invariant
6. opCmp
7. opEquals
8. toHash

## Once export is fixed will fix:

https://issues.dlang.org/show_bug.cgi?id=6019

https://issues.dlang.org/show_bug.cgi?id=23177

## Removal of workaround:

Allows to get rid of majority of manual exports in one of my 
DLL's.

https://github.com/Project-Sidero/basic_memory/blob/main/msvc_exports.def

There are some symbols that are from druntime (like atomics) that 
should be fixed with annotating them with export.

_D4core6atomic__T8atomicOpVAyaa2_2d3dTlTiZQzFNaNbNiNfKOliZl

_D4core8internal11destruction__T11__ArrayDtorTS6sidero4base10containers3map17concurrenthashmap__T21ConcurrentHashMapNodeTSQCxQCt4text7unicode13readonly_utf811String_UTF8TSQEuQEq8datetime4timeQHc4iana6TZFileZQEg6BucketZQHeFNbNiNfMAQHcZv



# RTInfo should be emitted when used not on declaration

RTInfo should be emitted when used, this prevents unresolved 
external symbols with -betterC dependencies.

## Fixes:

https://issues.dlang.org/show_bug.cgi?id=23820



# Thread crt constructor

This may only apply to Windows, but have a thread C runtime 
constructor/destructor will allow us to kill off SimpleDllMain.

DllMain is a wholly optional concept in C/C++ land. There is no 
reason for it to exist in D either.

crt_constructor/destructor already serves half of its purpose, 
only left is the per-thread behavior.

There will be some other behaviors that the compiler will need to 
switch from DllMain detection to performing when -shared is 
passed. To complete this transition.

## Completes:

https://issues.dlang.org/show_bug.cgi?id=23756



# Set object files file name to be the full module name with 
package

Linkers like to de-duplicate object files. To do this, the 
cheapest way is to compare file names.

Problem is dmd is using only the base name and not the full 
package + module name.

LDC supports this as opt-in, but I argue that this should be the 
default.

It also means if you use packages if you have two modules with 
the same base name they won't conflict.

## Completes:

https://issues.dlang.org/show_bug.cgi?id=3541