Speeding up importing Phobos files

[Seb]

On Friday, 7 June 2019 at 15:34:22 UTC, Mike Franklin wrote:
> On Friday, 7 June 2019 at 14:45:34 UTC, H. S. Teoh wrote:
>
>> The flip side of this coin is that having one file per module 
>> helps with code organization in typical use cases. As opposed 
>> to the chaotic situation in C/C++ where you can have one .h 
>> file with arbitrary numbers of .c/.cc files that implement the 
>> function prototypes, with no obvious mapping between them, or 
>> multiple .h files with a single amorphous messy .c/.cc file 
>> that implements everything. Or where there are arbitrary 
>> numbers of modules nested inside any number of .c/.cc files 
>> with no clear separation between them, and where filenames 
>> bear no relation with contents, making code maintenance 
>> challenging at best, outright nightmarish at worst.
>
> What I'm proposing is that a library's organization can be one 
> file per module while it is being developed, but once it is 
> published for consumption by others all packages/modules are 
> concatenated into one file so it's faster for users to import.  
> Or you could even concatenate some packages/modules, but not 
> others depending on how users typically interact with the 
> library.
>
> There may be use cases (like mine with the memory-mapped IO 
> registers) where users may want to actually develop with more 
> than one module per file.  Or when one wants to share 
> multi-package/module cut-and-pastable code examples. But those 
> are special cases.
>
> What's nice is that it changes nothing for users today.  It 
> just removes an arbitrary limitation to help improve import 
> speed while enabling some more flexibility for those who may it.
>
> Mike

Reading files is really cheap, evaluating templates and running 
CTFE isn't. That's why importing Phobos modules is slow - not 
because of the files it imports, but because of all the CTFE 
these imports trigger. This means we can do e.g.

- improve CTFE performance
- cache templates over multiple compilations (there's a DMD PR 
for this)
- make imports lazy
- reduce all the big CTFE bottlenecks in Phobos (e.g. std.uni)
- convert more Phobos code into templates with local imports to 
reduce the baseline import overhead

Ordered from hardest to easiest.