AST files instead of DI interface files for faster compilation and easier distribution

[Dmitry Olshansky]

On 13.06.2012 14:16, Iain Buclaw wrote:
> On 13 June 2012 10:45, Dmitry Olshansky<dmitry.olsh at gmail.com>  wrote:
>> On 13.06.2012 13:37, Iain Buclaw wrote:
>>>
>>> On 13 June 2012 09:07, Don Clugston<dac at nospam.com>    wrote:
>>>>
>>>> On 12/06/12 18:46, Walter Bright wrote:
>>>>
>>>>>
>>>>> On 6/12/2012 2:07 AM, timotheecour wrote:
>>>>>>
>>>>>>
>>>>>> There's a current pull request to improve di file generation
>>>>>> (https://github.com/D-Programming-Language/dmd/pull/945); I'd like to
>>>>>> suggest
>>>>>> further ideas.
>>>>>> As far as I understand, di interface files try to achieve these
>>>>>> conflicting goals:
>>>>>>
>>>>>> 1) speed up compilation by avoiding having to reparse large files over
>>>>>> and over.
>>>>>> 2) hide implementation details for proprietary reasons
>>>>>> 3) still maintain source code in some form to allow inlining and CTFE
>>>>>> 4) be human readable
>>>>>
>>>>>
>>>>>
>>>>> (4) was not a goal.
>>>>>
>>>>> A .di file could very well be a binary file, but making it look like D
>>>>> source enabled them to be loaded with no additional implementation work
>>>>> in the compiler.
>>>>
>>>>
>>>>
>>>> I don't understand (1) actually.
>>>>
>>>> For two reasons:
>>>> (a) Is lexing + parsing really a significant part of the compilation
>>>> time?
>>>> Has anyone done some solid profiling?
>>>>
>>>
>>> Lexing and Parsing are miniscule tasks in comparison to the three
>>> semantic runs done on the code.
>>>
>>> I added speed counters into the glue code of GDC some time ago.
>>>
>>> http://iainbuclaw.wordpress.com/2010/09/18/implementing-speed-counters-in-gdc/
>>>
>>> And here is the relavent report to go with it.
>>> http://iainbuclaw.files.wordpress.com/2010/09/d2-time-report2.pdf
>>>
>>>
>>> Example: std/xml.d
>>> Module::parse : 0.01 ( 0%)
>>> Module::semantic : 0.50 ( 9%)
>>> Module::semantic2 : 0.02 ( 0%)
>>> Module::semantic3 : 0.04 ( 1%)
>>> Module::genobjfile : 0.10 ( 2%)
>>>
>>> For the entire time it took to compile the one file (5.22 seconds) -
>>> it spent almost 10% of it's time running the first semantic analysis.
>>>
>>>
>>> But that was the D2 frontend / phobos as of September 2010.  I should
>>> re-run a report on updated times and draw some comparisons. :~)
>>>
>>
>> Is time spent on I/O accounted for in the parse step? And where is the rest
>> spent :)
>>
>
> It would be, the counter starts before the files are even touched, and
> ends after they are closed.

Ok, then parsing is indistinguishable from I/O and together are only 
tiny fraction of the whole. Great info, thanks.

>
> The rest of the time spent is in the GCC backend, going through the
> some 60+ code passes and outputting the assembly to file.
>

Damn, I like DMD :)



-- 
Dmitry Olshansky