Compile Imported Modules

[Jonathan Marler via Digitalmars-d]

On Friday, 25 August 2017 at 01:50:00 UTC, Jonathan Marler wrote:
> On Thursday, 24 August 2017 at 17:37:12 UTC, Jonathan Marler 
> wrote:
>> On Thursday, 24 August 2017 at 16:49:08 UTC, Seb wrote:
>>> On Thursday, 24 August 2017 at 16:32:32 UTC, Jonathan Marler 
>>> wrote:
>>>> [...]
>>>
>>> rdmd is really bad in terms of performance. If you call a 
>>> single D file with rdmd, it will always compile it twice. 
>>> There was an attempt to fix this 
>>> (https://github.com/dlang/tools/pull/194), but this has been 
>>> reverted as it introduced a regression and no one had time to 
>>> look at the regression.
>>> Moving rdmd into DMD has been on the TODO list for quite a 
>>> while and there is a consensus that the performance overhead 
>>> if rdmd isn't nice. However, IIRC there was no clear 
>>> consensus on how the integration should happen. I recall that 
>>> the plan was to do try this with "dmd as a library", but I'm 
>>> not sure whether that's really feasible ATM.
>>
>> Well this should solve the rdmd performance problem as well as 
>> make other user cases easier that don't necessarilly use rdmd.
>>
>> I had another thought that instead of making this an "opt-in" 
>> feature, it would probably make more sense to be an "opt-out" 
>> feature.  So by default the compiler would compile missing 
>> imported modules unless you indicate otherwise, maybe a 
>> command line switch like "-dont-compile-imports".  And I don't 
>> see how this would break anything.  Everything should work the 
>> same as it did before, it's just now you can omit imported 
>> module files from the command line and it should just work.
>
> I've looked through the DMD code to see how this could be 
> implemented and I've run into a problem.  The solution I came 
> up with was to go through all the imported modules and then 
> determine which ones need to be compiled that haven't been 
> given on the command line. The problem is, I don't know how to 
> determine whether a module was already compiled and given in an 
> obj/lib file.  For example,
>
> dmd something.obj anotherthing.lib prog.d
>
> As far as I know, the compiler has no idea which modules are 
> contained in "something.obj" and "anotherthing.lib".  It just 
> compiles the source given on then command line, then passes all 
> the object files and libraries to the linker, at which point 
> the concept of modules is lost.
>
> Am I correct in saying that the compiler has no idea which 
> modules an obj/lib file contains?

I created a prototype implementation here 
(https://github.com/dlang/dmd/pull/7099).

It uses the same logic that rdmd uses to determine if a module 
exists in a given object/library file.  Pretty cool that now I 
can compile any code without having to list the import module 
files!

Instead of:

dmd prog.d -Isomelib somelib\foo\module1.d somelib\foo\module2.d 
somelib\foo\module3.d somelib\foo\module4.d somelib\foo\module5.d 
somelib\foo\module6.d -Ianotherlib anotherlib\bar\module1.d 
anotherlib\bar\module2.d anotherlib\bar\module3.d 
anotherlib\bar\module4.d anotherlib\bar\module5.d

I can do:

dmd -ci prog.d -Isomelib -Ianotherlib