DI File Generation

[Jonathan M Davis]

On Wednesday, December 14, 2011 10:47:17 Adam Wilson wrote:
> Hello everyone,
> 
> As some of you may be aware I have been working on improvements to the .di
> file generation capabilities in D and I've run across a few things that I
> think should be put to the community.
> 
> The first involves the handling of variable instantiation. Say you create
> the following member variable in a class:
> 
> export int Foo = 1;
> 
> Currently the di generation output is identical to the above line. My
> understanding is that, similar to C/C++, D doesn't care if you keep the
> initializer in the include file so this is primarily a stylistic question.
> Should we keep the output as is, or remove the initializer? My personal
> preference is to remove the initializer but I think the community should
> decide.
> 
> Another question that has come up pertains to the reason for the original
> implementation of DI generation, inlining. According to what I've read on
> these newsgroups and various other sources, DI generation is supposed to
> include the implementation for some small functions that would be good
> inlining candidates, however, it looks like is actually happening is that
> all DMD does is check to see if there are any cases where the compiler
> would fail if it attempted to inline the function. As long as the compiler
> can compile it, it gets thrown into the DI file, regardless of size. My
> current patches never include the implementation, and I think that for
> most people this is a reasonable solution, if not ideal. All of the people
> I've talked to seemed to agree. What do you think? Does this work for you?
> Should there be a way to specify the maximum size of an inline function?
> NOTE: Doing this would require at least a command line switch and probably
> some significant reworking of the function symbol exporter. As such it'd
> take a lot more time to get into to D; I'd recommend implementing the
> simple solution first then adding this only if it's requested by a
> significant number of people.
> 
> The next question is primarily directed at Walter, but anybody who knows
> anything about the compiler internals is welcome to chime in. For
> functions defined in Class and Interfaces, calling isFuncDeclaration() as
> I iterate through the symbol tree works as expected and returns the
> Function. However, when I try to call that on a function that is declared
> in a module (with no surrounding class) or in a struct, it always returns
> null, regardless of whether or not it's actually a function. Is there any
> way to determine if the symbol is a function when it's not a class member?
> NOTE: As near as I can tell, it is actually a FuncDeclaration.
> 
> The final bit of DI generation revolves around the handling of protected
> functions in exported classes. But that's better suited to a separate
> topic as that involves the general design of scoping in D2 and isn't
> limited to DI generation. I'll be posting another topic on that soon.
> 
> Thank you all for taking the time to read and chime in on this!

Considering that you must have the bodies of functions to do either inlining 
or CTFE, stripping _anything_ from a .di file carries a definite cost. Now, 
maybe large functions shouldn't be put in there, but certainly anything which 
could be reasonably inlined should be, and depending on what you're doing with 
CTFE, you're going to need the big ones anyway - though that depends on the 
particular usage more than inlining does, so having to add the function bodies 
manually for CTFE isn't as big a deal as it would be for inlining.

Also, _all_ templated _anything_ has to be put in the .di file, and a lot of 
stuff tends to be templated in D (especially in the standard library), so it's 
pretty easy to need to put a large portion of your code in the .di file anyway.

Personally, I think that .di files aren't a great idea in general, precisely 
because of the issues with inlining and CTFE, so I never use them. They 
generally complicate things needlessly. In some cases, it probably does make 
sense to use them (e.g. you have a very large project that needs the 
additional boost in compilation speed or if you have a number of classes with 
large function bodies which aren't going to be used with CTFE and can't be 
inlined, because they're virtual), but in general, I just don't think that 
they're worth messing with - especially when you consider having to keep the 
.di and .d files in sync.

dmd is very fast compiler, and I really think that in the majority of cases, 
there's not really any point to .di files. And when you _do_ use them, a lot of 
code has to be in them anyway, or you lose the ability to inline or use CTFE, 
which would harm performance and impede what your code can do. And if you're 
using templates much, then it all ends up in the .di file  anyway.

- Jonathan M Davis