Speeding up importing Phobos files

Temtaime temtaime at gmail.com
Sat Jan 19 09:00:38 UTC 2019


On Saturday, 19 January 2019 at 08:45:27 UTC, Walter Bright wrote:
> Andrei and I were talking on the phone today, trading ideas 
> about speeding up importation of Phobos files. Any particular D 
> file tends to import much of Phobos, and much of Phobos imports 
> the rest of it. We've both noticed that file size doesn't seem 
> to matter much for importation speed, but file lookups remain 
> slow.
>
> So looking up fewer files would make it faster.
>
> Here's the idea: Place all Phobos source files into a single 
> zip file, call it phobos.zip (duh). Then,
>
>     dmd myfile.d phobos.zip
>
> and the compiler will look in phobos.zip to resolve, say, 
> std/stdio.d. If phobos.zip is opened as a memory mapped file, 
> whenever std/stdio.d is read, the file will be "faulted" into 
> memory rather than doing a file lookup / read. We're 
> speculating that this should be significantly faster, besides 
> being very convenient for the user to treat Phobos as a single 
> file rather than a blizzard. (phobos.lib could also be in the 
> same file!)
>
> It doesn't have to be just phobos, this can be a general 
> facility. People can distribute their D libraries as a zip file 
> that never needs unzipping.
>
> We already have https://dlang.org/phobos/std_zip.html to do the 
> dirty work. We can experiment to see if compressed zips are 
> faster than uncompressed ones.
>
> This can be a fun challenge! Anyone up for it?
>
> P.S. dmd's ability to directly manipulate object library files, 
> rather than going through lib or ar, has been a nice success.

C'mon, everyone has a SSD, OS tends to cache previously opened 
files. What's the problem ?
Better speedup compilation speed.


More information about the Digitalmars-d mailing list