Possible solution to template bloat problem?

[H. S. Teoh]

On Mon, Aug 19, 2013 at 10:37:35PM +0200, Dicebot wrote:
> Just two words: "separate compilation".
> 
> Any solution that is going to address template problem needs to
> improve current state with such compilation model, not make it even
> worse.

I thought about that, actually. (I'm a fan of separate compilation,
though the current state of D makes it more advantageous to compile
everything in one go where possible.)

There should be *one* static library where template instantiations are
dumped. Let's call it templates.a for lack of a better name. Inside this
archive, template instantiations are organized according to the module
hierarchy -- i.e., if you import std.stdio and use writeln, the
instantiation of writeln will be put into std/stdio/writeln_*.o in
templates.a, NOT the root module.

Furthermore, if a particular instantiation is already present in
templates.a, then it is not duplicated, but simply replaced (I was going
to say ignored, but replace is better in case templates.a contains
leftovers from a previous compilation run). This eliminates duplicate
instantiations of the same templates in multiple modules.


> As an alternative, I have proposed one of two approaches (or both):
> 
> 1) Stop dumping all symbols into root module supplied from the
> command line. Emit symbols to object files that match modules they
> were instantiated from. If symbol has no valid source point (==
> constraint or CTFE) than don't emit it at all.

+1.

Plus, DMD really needs to stop stripping paths from object files *by
default* -- now that we have package.d, this will blow up in ugly ways
once you have templates in package.d that get instantiated and there are
multiple packages with package.d.


> 2) Create object files in format that allows usage of `ld
> --gc-sections` (garbage collection of unused symbols upon linking).
> Don't know if similar thing exists for Windows.

Well, the reason I went for bare linker essentials was precisely to
avoid platform-specific issues like this. But OTOH, on platforms that
support the equivalent of --gc-sections, we could just emit template
instantiations into separate sections instead of separate modules in a
static library. The static library approach can serve as a fallback on
platforms that don't support this.


> Latter should be relatively easy to do but it is not cross-platform
> and it does not help build systems with tracking rebuild conditions.
> 
> Former feels like a proper approach and I have been working on it
> (also @eskimor) for some time. But it is rather hard as relevant
> code does not seem to track required information at all and probably
> no one but Walter knows all minor details about its design. To sum
> it up - I have not been able to produce a pull request that passes
> the test suite so far (though I have put it on hold for some time,
> going to return to this).

Hmm. It'd be nice if this could be made to work. One thing I'm not very
happy with in D is the sheer size of the executables even for
relatively simple programs. If unused symbols could be stripped during
linking, this would help a lot.


T

-- 
Political correctness: socially-sanctioned hypocrisy.