Big picture on shared libraries when they go wrong, how?

[Atila Neves]

On Friday, 10 May 2024 at 15:21:15 UTC, Richard (Rikki) Andrew 
Cattermole wrote:
> On 11/05/2024 1:27 AM, Atila Neves wrote:
>>>> "you cannot tell the compiler that a module is not in your 
>>>> binary." - isn't this exactly what happens with `export` on 
>>>> a declaration (as opposed to a definition)? That is, my 
>>>> understanding is that `export` with no body means 
>>>> `dllimport` and `export` with a body means `dllexport`.
>>>
>>> Keep in mind that nobody that I know of is using it to mean 
>>> this today.
>> 
>> Maybe the recommendation should then be that they should? 
>> Doesn't the point still stand that "you cannot tell the 
>> compiler that a module is not in your binary" isn't actually 
>> true? I saw in one issue where there was a problem with 
>> variable declaration though, where dllimport/dllexport was 
>> determined by the presence or not of an initialiser, which 
>> is... yuck.
>
> "you cannot tell the compiler that a module is not in your 
> binary" is true, there is no syntax or cli flag to do this 
> today.

Correct, sorry, I thought we were talking about symbols and 
somehow missed "module". The question would then be: why would 
someone want to tell the compiler that an entire module is out of 
binary?

> But yes having an initializer or not, should not determine the 
> symbol mode. I did my best to clean it up in a way that would 
> keep things simple and not break the world.
>
> This is why I've simplified things down to:
>
> Use ``export`` + ``extern`` to go into ``DllImport`` mode.

Makes sense.

> Or rely on the external import path switch to set the 
> ``extern`` for the majority of users. Which is ideal for things 
> like the di generator or build managers ;)

Do you mean "build systems"?

> Or use the dllimport override switch to set all symbols found 
> from a module that is known to be out of binary as 
> ``DllImport`` (helps with mixing some imports being in binary 
> and some out).
>
> Yeah that's not the best example, but keep in mind that symbol 
> is not in ``DllImport`` mode, its ``Internal``.
>
> And that right there is a problem.

And wouldn't the solution be to add `export`?

>> Why would the entire module be dllimport?
>
> Walter came up with this idea a while back, so I'm a tad 
> defensive towards it.
>
> https://github.com/dlang/dmd/pull/15298

I went through all of that and am still confused as to whether 
you want or don't want whole modules to be declared out of 
binary, or why one would want to do that.

> With Walter's idea the ``app`` module constructor could be 
> called before ``binding``'s does, which absolutely should not 
> happen.

I don't know why not, nor did I understand the relevance of the 
example.

> Basically a module can have some symbols out of binary whilst 
> the rest are in. So you can't make this sort of assumption.

Ok.

>> My question is: when would I want to export a private symbol?
>
> Q1: Should a given symbol be private, yes?
> Q2: Does any template use the previous symbol?
>
> A: That is why you would export a private symbol.

Can't we do what C++ does and stick the private symbol where the 
template is being instantiated?

> ```d
> module database_access;
>
> @safe:
>
> void doAThing(T)() {
> 	iCanKillYourDatabase(false, T.sizeof);
> }
>
>
> /*private:*/
> export:
>
> void iCanKillYourDatabase(bool doWrongThing = true, size_t 
> sizeOfThing) {
> 	if (doWrongThing)
> 		database.corruptSilently();
> }
> ```
>
> A bit dramatic (due to unrealistic nature), but that should get 
> the point across that ``iCanKillYourDatabase`` should be 
> private but also exported.

Inline it instead? The code is right there.

> For generated symbols like ``T.init``, ``opCmp``, 
> ``ModuleInfo`` ext., we have zero control over these currently. 
> Either you use a linker script or you use negative annotation. 
> Everyone I know of uses negative annotation (although I support 
> both).

What would the solution be?

>>>> "By not exporting ModuleInfo and assuming it is available 
>>>> the compiler introduces a hidden dependency on a generated 
>>>> symbol that may not exist." - do we have an issue for that? 
>>>> I searched for ModuleInfo in the issues but none of them 
>>>> looked like a match to me.
>>>
>>> Yes two. They are referenced in the article.
>>>
>>> Note: they are not duplicates.
>>>
>>> Okay I lie there is a bunch more.
>> 
>> Thanks!
>> 
>> On a somewhat related note, we use dlls at work and seem to 
>> have fixed "everything" by using ldc and `-fvisibility=hidden 
>> -dllimport=defaultLibsOnly`, as well as 
>> `-link-defaultlib-shared`.
>
> ``-link-defaultlib-shared`` sets the druntime to be a shared 
> library (which lets face it should be the default).

Given how much people go on about how great Go is because it 
links statically (even though C/C++ have been able to do this for 
basically forever if you opt-in), I'm not sure of that.

> ``-dllimport=defaultLibsOnly`` all symbols for druntime and 
> phobos are defaulting to ``DllImport`` but none others.
>
> As for ``-fvisibility=hidden``, that would imply that you are 
> using positive annotation in every code base. Which is curious 
> considering Martin's prior work has been the exact opposite to 
> this.

I grepped for `export` and it appears in quite a few places as 
expected.

> Perhaps the one you are referring to is a plugin with a known 
> fixed public API? That being positive annotation and everything 
> else being compiled in being hidden would make sense.

I think that's what we're doing yes, and that the fact that we 
are is a good thing.