AST files instead of DI interface files for faster compilation and easier distribution

[Iain Buclaw]

On 13 June 2012 10:45, Dmitry Olshansky <dmitry.olsh at gmail.com> wrote:
> On 13.06.2012 13:37, Iain Buclaw wrote:
>>
>> On 13 June 2012 09:07, Don Clugston<dac at nospam.com>  wrote:
>>>
>>> On 12/06/12 18:46, Walter Bright wrote:
>>>
>>>>
>>>> On 6/12/2012 2:07 AM, timotheecour wrote:
>>>>>
>>>>>
>>>>> There's a current pull request to improve di file generation
>>>>> (https://github.com/D-Programming-Language/dmd/pull/945); I'd like to
>>>>> suggest
>>>>> further ideas.
>>>>> As far as I understand, di interface files try to achieve these
>>>>> conflicting goals:
>>>>>
>>>>> 1) speed up compilation by avoiding having to reparse large files over
>>>>> and over.
>>>>> 2) hide implementation details for proprietary reasons
>>>>> 3) still maintain source code in some form to allow inlining and CTFE
>>>>> 4) be human readable
>>>>
>>>>
>>>>
>>>> (4) was not a goal.
>>>>
>>>> A .di file could very well be a binary file, but making it look like D
>>>> source enabled them to be loaded with no additional implementation work
>>>> in the compiler.
>>>
>>>
>>>
>>> I don't understand (1) actually.
>>>
>>> For two reasons:
>>> (a) Is lexing + parsing really a significant part of the compilation
>>> time?
>>> Has anyone done some solid profiling?
>>>
>>
>> Lexing and Parsing are miniscule tasks in comparison to the three
>> semantic runs done on the code.
>>
>> I added speed counters into the glue code of GDC some time ago.
>>
>> http://iainbuclaw.wordpress.com/2010/09/18/implementing-speed-counters-in-gdc/
>>
>> And here is the relavent report to go with it.
>> http://iainbuclaw.files.wordpress.com/2010/09/d2-time-report2.pdf
>>
>>
>> Example: std/xml.d
>> Module::parse : 0.01 ( 0%)
>> Module::semantic : 0.50 ( 9%)
>> Module::semantic2 : 0.02 ( 0%)
>> Module::semantic3 : 0.04 ( 1%)
>> Module::genobjfile : 0.10 ( 2%)
>>
>> For the entire time it took to compile the one file (5.22 seconds) -
>> it spent almost 10% of it's time running the first semantic analysis.
>>
>>
>> But that was the D2 frontend / phobos as of September 2010.  I should
>> re-run a report on updated times and draw some comparisons. :~)
>>
>
> Is time spent on I/O accounted for in the parse step? And where is the rest
> spent :)
>

It would be, the counter starts before the files are even touched, and
ends after they are closed.

The rest of the time spent is in the GCC backend, going through the
some 60+ code passes and outputting the assembly to file.


-- 
Iain Buclaw

*(p < e ? p++ : p) = (c & 0x0f) + '0';