AST files instead of DI interface files for faster compilation and easier distribution

[Iain Buclaw]

On 16 June 2012 10:18, Iain Buclaw <ibuclaw at ubuntu.com> wrote:
> On 13 June 2012 12:47, Iain Buclaw <ibuclaw at ubuntu.com> wrote:
>> On 13 June 2012 12:33, Kagamin <spam at here.lot> wrote:
>>> On Wednesday, 13 June 2012 at 11:29:45 UTC, Kagamin wrote:
>>>>
>>>> The measurements should be done for modules being imported, not the module
>>>> being compiled.
>>>> Something like this.
>>>> ---
>>>> import std.algorithm;
>>>> import std.stdio;
>>>> import std.typecons;
>>>> import std.datetime;
>>>>
>>>> int ok;
>>>> ---
>>>
>>>
>>> Oh and let it import .d files, not .di
>>
>> std.datetime is one reason for me to run it again. I can imagine that
>> *that* module will have an impact on parse times.  But I'm still
>> persistent that the majority of the compile time in the frontend is
>> done in the first semantic pass, and not the read/parser stage. :~)
>>
>>
>
> Rebuilt a compile log with latest gdc as of writing on the 2.059
> frontend / library.
>
> http://iainbuclaw.files.wordpress.com/2012/06/d2time_report32_2059.pdf
> http://iainbuclaw.files.wordpress.com/2012/06/d2time_report64_2059.pdf
>
>
> Notes about it:
> - GCC has 4 new time counters
>  -  phase setup  (time spent loading the compile time environment)
>  -  phase parsing  (time spent in the frontend)
>  -  phase generate (time spent in the backend)
>  -  phase finalize  (time spent cleaning up and exiting)
>
> - Of the phase parsing stage, it is broken down into 5 components
>  -  Module::parse
>  -  Module::semantic
>  -  Module::semantic2
>  -  Module::semantic3
>  -  Module::genobjfile
>
> - Module::read, Module::parse and Module::importAll in the one I did 2
> years ago are now counted as part of just the one parsing stage,
> rather than separate just to make it a little bit more balanced. :-)
>
>
> I'll post a tl;dr later on it.
>

tl;dr

Total number of source files compiled: 207
Total time to build druntime and phobos:  78.08 seconds
Time spent parsing: 17.15 seconds
Average time spent parsing: 0.08 seconds
Time spent running semantic passes: 10.04 seconds

Time spent generating backend AST: 2.15 seconds
Time spent in backend: 48.62 seconds


So parsing time has taken quite a hit since I last did any reports on
compilation speed of building phobos.  I suspect most of that comes
from the loading of symbols from all imports and that there have been
some large additions to phobos recently which provide a constant
bottle neck if one was to choose compiling one source at a time.  As
the apparent large amount of time spent parsing sources does not show
when compiling all at once.

 Module::parse: 0.58 seconds (1%)
 Module::semantic: 0.24 seconds (1%)
 Module::semantic2: 0.01 seconds (0%)
 Module::semantic3: 2.85 seconds (6%)
 Module::genobjfile: 1.24 seconds ( 3%)
 TOTAL: 47.06 seconds

Considering that the entire phobos library is some 165K lines of code,
I don't see why people aren't laughing about just how quick the
frontend is at parsing. :~)


Regards
-- 
Iain Buclaw

*(p < e ? p++ : p) = (c & 0x0f) + '0';