Lib change leads to larger executables

Pragma ericanderton at yahoo.removeme.com
Thu Mar 8 13:48:36 PST 2007


Don Clugston wrote:
> Pragma wrote:
>> Sean Kelly wrote:
>>> kris wrote:
>>>> Walter Bright wrote:
>>>>> kris wrote:
>>>>>
>>>>>> Walter Bright wrote:
>>>>>>
>>>>>>> Sure, but in this particular case, it seems that "core" is being 
>>>>>>> imported without referencing code in it. The only reason the 
>>>>>>> compiler doesn't generate the char[][] TypeInfo is because an 
>>>>>>> import defines it. The compiler does work on the assumption that 
>>>>>>> if a module is imported, then it will also be linked in.
>>>>>>
>>>>>> This core module, and the entire locale package it resides in, is 
>>>>>> /not/ imported by anything. I spelled that out clearly before. 
>>>>>> You're making an assumption it is, somehow ... well, it is not. 
>>>>>> You can deduce that from the fact that the link succeeds perfectly 
>>>>>> well without that package existing in the library.
>>>>
>>>> After taking a much needed break from this, I'm having another bash 
>>>> at it. The locale package highlighted in the last bout has been 
>>>> through a number of changes, and thus the behaviour will now be 
>>>> somewhat different than before.
>>>>
>>>> For a refresher on this issue, here's an overview: 
>>>> http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D&article_id=49257 
>>>>
>>>>
>>>> The upshot is that reordering the content as presented to the 
>>>> librarian now has an effect on the resultant binary size. This tells 
>>>> me two things:
>>>>
>>>> 1) there were more than just the one compiler-generated unresolved 
>>>> symbol in the first case, though I did not spot it after many 
>>>> painstaking hours of frustrating effort. In fact, there may well 
>>>> have been a number of vaguely intertwined dependencies spread 
>>>> throughout the library, due entirely to these compiler-generated 
>>>> symbols.
>>>>
>>>> 2) there is no feasible manner in which a developer can control how 
>>>> lib contents are linked while the compiler continues to generate 
>>>> duplicate symbols under the covers. The fact that it does this on a 
>>>> regular basis simply serves to highlight a critical problem.
>>>>
>>>>
>>>>  > Then the typeinfo for char[][] is being generated by another 
>>>> module. I
>>>>  > suggest it would be helpful to find that module. Grep is a handy 
>>>> tool
>>>>  > for finding it.
>>>>
>>>> What would be the point? These symbols are compiler generated, and 
>>>> exist in a variety of places because of that fact alone. Lest we 
>>>> forget, we're not even talking about just one symbolic name ~ the 
>>>> compiler generates duplicate symbols for any typeinfo that does not 
>>>> match the pre-packaged list (such as char[][]). The end result is a 
>>>> potential rats-nest of duplicate symbols, creating a maze of 
>>>> intricate and fragile dependencies across the lib itself.
>>>>
>>>> To put things into perspective, I have absolutely no way of knowing 
>>>> what the real dependencies within this library actually are. As 
>>>> such, I'd have to describe the situation as being "out of control". 
>>>> This is wholly due to the duplicate compiler-generated symbols.
>>>
>>> It's a long-term proposition, but what about delaying the generation 
>>> of TypeInfo until link-time?  The MS linker already optionally 
>>> performs code generation to allow for more optimized executables, so 
>>> I assume such a thing is definitely possible.  It would obviously 
>>> require a new linker, but I don't see any other way to address these 
>>> "silent dependency" issues, etc.  If I had the time I'd try it out, 
>>> but as things stand there's no way that could happen before September.
>>
>> I was going to say: new linker == our problem.  I'm in the same boat 
>> on the no time thing, but maybe we can get a group effort going later 
>> on.  If for no other reason, we could investigate what a native 64-bit 
>> toolchain for D on Windows might look like.
> 
> I've been wondering how far your work with DDL goes towards writing a 
> linker? Certainly the work you've done with making sense of the OMF and 
> ELF specs, and parsing the obj files, seems to be a huge chunk of the 
> task. A lot of code would be common to both tasks, surely?

Well, the OMF loader needs some polish and some subtle refactoring (read:Tango) but I have thought the same myself.  The 
big stumbling block (for me at least) is understanding the "E" in "PE/COFF" for emitting .exe and .dll files.  The 
intermediate part, matching dependencies, is really kind of simple only up until you get into various forms of 
optimization: culling unused segments & symbols, deep dependency analysis, *fast* linking, minimize size, optimize for 
speed, etc.  Typing "link /?" in your console will quickly cast a very large shadow over this area.

ELF is another issue.  Flectioned is light-years ahead of DDL on that front.  But combined with a (future) upgraded DDL, 
we'll pretty much have "libtools for D".

Now if we had a library that allowed for reading *and* writing of COFF files per 100% of the specification, then I'd 
imagine that this wouldn't be too far out of reach.

-- 
- EricAnderton at yahoo



More information about the Digitalmars-d mailing list