Speeding up importing Phobos files

Walter Bright newshound2 at digitalmars.com
Sat Jan 19 08:45:27 UTC 2019


Andrei and I were talking on the phone today, trading ideas about speeding up 
importation of Phobos files. Any particular D file tends to import much of 
Phobos, and much of Phobos imports the rest of it. We've both noticed that file 
size doesn't seem to matter much for importation speed, but file lookups remain 
slow.

So looking up fewer files would make it faster.

Here's the idea: Place all Phobos source files into a single zip file, call it 
phobos.zip (duh). Then,

     dmd myfile.d phobos.zip

and the compiler will look in phobos.zip to resolve, say, std/stdio.d. If 
phobos.zip is opened as a memory mapped file, whenever std/stdio.d is read, the 
file will be "faulted" into memory rather than doing a file lookup / read. We're 
speculating that this should be significantly faster, besides being very 
convenient for the user to treat Phobos as a single file rather than a blizzard. 
(phobos.lib could also be in the same file!)

It doesn't have to be just phobos, this can be a general facility. People can 
distribute their D libraries as a zip file that never needs unzipping.

We already have https://dlang.org/phobos/std_zip.html to do the dirty work. We 
can experiment to see if compressed zips are faster than uncompressed ones.

This can be a fun challenge! Anyone up for it?

P.S. dmd's ability to directly manipulate object library files, rather than 
going through lib or ar, has been a nice success.


More information about the Digitalmars-d mailing list