Speeding up importing Phobos files

[Mike Franklin]

On Saturday, 19 January 2019 at 08:45:27 UTC, Walter Bright wrote:
> Andrei and I were talking on the phone today, trading ideas 
> about speeding up importation of Phobos files. Any particular D 
> file tends to import much of Phobos, and much of Phobos imports 
> the rest of it. We've both noticed that file size doesn't seem 
> to matter much for importation speed, but file lookups remain 
> slow.

The topic of import speed when an implementation is spread over 
multiple files came up again on a PR I'm working on, so I wanted 
to share an idea I had.

Why not remove the arbitrary limitation that a module or package 
must be tied to a file or directory respectively?  That is within 
one file you could have something like this:

---
package thePackage
{
     module thePackage.module1
     {
         ...
     }

     module thePackage.module2
     {
         ...
     }
}

package thePackage2
{
     ....
}
---

Then when one distributes their library, they could concatenate 
all the files into one, potentially reducing the overhead 
required to querying the filesystem.  After loading the file, the 
entire library would essentially be cached in memory.

It would also help with a number of other helpful patterns.  For 
example, I currently use the following pattern for creating 
register maps of memory-mapped IO:

---
final abstract class MyRegisterBank
{
     final abstract class MyRegister
     {
        //static properties for bit-fields
     }
}
// See 
https://github.com/JinShil/stm32f42_discovery_demo/blob/master/source/stm32f42/spi.d for the real thing
---

I could avoid doing silly things like that and use modules in a 
single file instead, which would be a more appropriate model for 
that use case.  I currently don't use modules simply because I'd 
end up with 100's of file to manage for a single MCU.

Another trivial use case would be the ability to test multiple 
packages and modules on run.dlang.io without the need for any 
additional infrastructure.

I'm sure the creativity of this community would find other 
interesting uses cases.

It seems like an arbitrary limitation to have the package/module 
system hard-coded to the underlying storage technology, and 
removing that limitation may help with the import speed while 
also enabling a few interesting use cases.

Mike