Who can make Phobos faster to import?

Joakim dlang at joakim.fea.st
Thu Dec 21 03:31:16 UTC 2017


On Wednesday, 20 December 2017 at 18:21:33 UTC, Andrei 
Alexandrescu wrote:
> On 12/15/2017 02:10 PM, Seb wrote:
>> Following the discussion at 
>> https://github.com/dlang/phobos/pull/5916, we realized that 
>> there's quite some overhead in importing some Phobos modules. 
>> Ideally importing a Phobos module without using it, should be 
>> cost-free (i.e. pay for what you use).
>> 
>> Test case: consider a file with `import std.X` and compile it 
>> with `dmd -c -o-`.
>> 
>> A quick overview of the worst of the worst:
>> 
>> std.regex    1.03s
>> std.net.curl    0.28s
>> std.concurrency    0.23s
>> std.encoding    0.18s
>> std.zip            0.16s
>> std.path    0.14s
>> std.file    0.14s
>> std.mmfile    0.14s
>> std.datetime    0.14s
>> std.socket    0.12s
>> std.string    0.10s
>> std.uni            0.09s
>> std.process    0.05s
>> 
>> This following discussions have already brought a huge 
>> improvements to its light:
>> 
>> https://github.com/dlang/phobos/pull/5931
>> 
>> Can you do a another one?
>
> Dmitry wrote a nice PR for that, and I wrote two:
>
> https://github.com/dlang/phobos/pull/5942
> https://github.com/dlang/phobos/pull/5931
> https://github.com/dlang/phobos/pull/5948
>
> My approach has been the following:
>
> * First I tried to clarify which module imports what by means 
> of replacing "import std.xxx;" with "import std.xxx : yyy, zzz, 
> ...;" That has been of tremendous help in assessing each 
> module's dependency liability.
>
> * Then I tried to figure which imports are most costly. I 
> inserted __EOF__ in the module at different parts and measure 
> how long it takes to just import that module. After a few 
> attempts it becomes clear which imports are heaviest.
>
> * Then I pushed those heavy imports down where they're needed. 
> An import placed in a function will not be opened while that 
> function's module gets imported. Same goes about template 
> types, but not structs or classes. (That should be fixed; see 
> Seb's comment in my last PR.)
>
> Not only such work reduces import times, but it's illuminating 
> for maintenance because it represents dependencies in a simple, 
> fine-grained matter.
>
> A tool (call it depend - heh) to automate that would be 
> awesome. For example, this run would make all imported names 
> explicit:
>
> depend --explicit *.d
>
> This run would push all imports down to the innermost scope of 
> usage:
>
> depend --pushdown *.d
>
>
> Andrei

I'm on it.  I tried using a version of Seb's frontend library 
yesterday but with the standard dmd main and tried running it on 
std.stdio, but it asserted in asmSemantic because of an asm block 
somewhere in druntime, so I'm back to using a tweaked full dmd 
with the backend simply disabled again:

https://github.com/dlang/dmd/pull/7425
https://github.com/dlang/dmd/blob/master/src/dmd/gluelayer.d#L47
https://gist.github.com/joakim-noah/09cf49bee3d82b03a54f

Once I have something basic working, I'll put it up on github, so 
others who are interested can play with it and pitch in to make 
it a real tool.


More information about the Digitalmars-d mailing list