Speeding up importing Phobos files

[Steven Schveighoffer]

On 1/19/19 3:32 PM, Neia Neutuladh wrote:
> On Sat, 19 Jan 2019 11:56:29 -0800, H. S. Teoh wrote:
>> Excellent finding! I *knew* something was off when looking up a file is
>> more expensive than reading it.  I'm thinking a quick fix could be to
>> just cache the intermediate results of each lookup, and reuse those
>> instead of issuing another call to opendir() each time.  I surmise that
>> after this change, this issue may no longer even be a problem anymore.
> 
> It doesn't even call opendir(). It assembles each potential path and calls
> exists(). Which might be better for only having a small number of imports,
> but that's not the common case.
> 
> I've a partial fix for Posix, and I'll see about getting dev tools running
> in WINE to get a Windows version. (Which isn't exactly the same, but if I
> find a difference in FindFirstFile / FindNextFile between Windows and
> WINE, I'll be surprised.)
> 
> I'm not sure what it should do when the same module is found in multiple
> locations, though -- the current code seems to take the first match. I'm
> also not sure whether it should be lazy or not.
> 
> Also symlinks and case-insensitive filesystems are annoying.
> 
> https://github.com/dhasenan/dmd/tree/fasterimport
> 

I wonder if packages could be used to eliminate possibilities.

For example, std is generally ONLY going to be under a phobos import 
path. That can eliminate any other import directives from even being 
tried (if they don't have a std directory, there's no point in looking 
for an std.algorithm directory in there).

Maybe you already implemented this, I'm not sure.

I still find it difficult to believe that calling exists x4 is a huge 
culprit. But certainly, caching a directory structure is going to be 
more efficient than reading it every time.

-Steve