Lib change leads to larger executables

[Sean Kelly]

kris wrote:
> 
> 6) The dependency that /does/ cause the problem is one generated by the 
> D compiler itself. It generates and injects false 'dependencies' across 
> object modules. Specifically, that's effectively how the linker treats 
> them.
...
> 9) The Fake dependencies cause the linker to pick up and bind whatever 
> module happens to satisfy it's need for the typeinfo resolution. In this 
> case, the linker sees Core.obj with a char[][] decl exposed, so it say 
> "hey, this is the place!" and binds it. Along with everything else that 
> Core.obj actually requires.

This is the crux of the problem.  In C/C++, problem areas can typically 
be identified and addressed.  In D however, the problem areas are 
related to "hidden" data and may manifest differently for different 
applications.  They can still be identified and addressed, but the 
process is far more brittle, as any code change can have cascading 
effects on application size.

Still, I don't entirely understand why this appears to not be an issue 
using Build, which has historically had bloat issues in some cases.  Was 
it just luck, or do things actually change when objects are stored in a 
library as opposed to not?

> Case in point: you have to strip the library down by hand, and very very 
> carefully sift through the symbols and literally hundreds of library 
> builds until you finally get lucky enough to stumble over the problem.
> 
> Walter asserts that the linker can be tricked into doing the right 
> thing. This seems to show a lack of understanding on his part about the 
> problem and the manner in which the lib and linker operate.

So far, I see two options for using Win32 libraries: the old way, which 
created relatively lean EXEs but had link errors for template code, or 
the new way, which requires the laborious process outlined above.

Interestingly, eschewing libraries in favor of object-level linking has 
always worked and doesn't seem to exhibit either of the above problems 
(as I mentioned above).  As much as people have been pushing for working 
D libraries, given the above alternatives I'm somewhat inclined to stick 
with Build unless I'm integrating with a C application.

By the same token none of these problems appear to have ever existed on 
Linux, be it because of the ELF format, the 'ld' linker, or some other 
confluence of planetary alignment and sheer luck.  Can anyone confirm 
that this is indeed true?

> As was pointed out to me, OMF librarians actually uses a two-level 
> hashmap to index the library entries. This is used by the linker to 
> locate missing symbols. I think it's clear that this is not a linear 
> lookup mechanism as had been claimed, and is borne out by experiments 
> that show the process cannot be controlled, and the linker cannot be 
> faked in usable or dependable manner.

This may not be true of optlink however.  I suspect the hashmap is 
probably more likely of linkers that do segment-level linking?  There 
seems little point in the complexity otherwise.

> I feel it important to point out that this powerful I18N package is, in 
> no way, at fault here. The D compiler simply injected the wrong symbol 
> into the wrong module at the wrong time, in the wrong order, and the 
> result is that this package gets linked when it's not used or imported 
> in any fashion by the code design. Instead, the dependency is created 
> entirely by the compiler. That's a problem. It is a big problem. And it 
> is a problem every D developer will face, at some point, when using DM 
> tools. But it is not a problem rooted in the Tango code.

Agreed.  Tango was merely one of the first to encounter it because it's 
one of the first "large" D libraries.


Sean