[spec] Phases of translation

[Dibyendu Majumdar]

On Saturday, 1 June 2019 at 13:34:07 UTC, Dibyendu Majumdar wrote:
> On Tuesday, 21 May 2019 at 13:24:11 UTC, Dibyendu Majumdar 
> wrote:
>>> On Monday, 20 May 2019 at 16:13:39 UTC, Walter Bright wrote:
>>>
>>>> The crucial thing to know is that the tokenizing is 
>>>> independent of parsing, and parsing is independent of 
>>>> semantic analysis.
>>>>
>>>
>>> I am trying to understand this aspect - I found the small 
>>> write up in the Intro section not very clear. Would be great 
>>> if you could so a session on how things work - maybe video 
>>> cast?
>>>
>>> For example, the mixin declaration has to convert a string to 
>>> AST I guess? When does this happen? Does it not need to 
>>> invoke the lexer on the generated string and build AST while 
>>> already in the semantic stage?
>>>
>>
>> Currently the Intro says:
>>
>> The process of compiling is divided into multiple phases. Each 
>> phase has no dependence on subsequent phases. For example, the 
>> scanner is not perturbed by the semantic analyzer. This 
>> separation of the passes makes language tools like syntax 
>> directed editors relatively easy to produce. It also is 
>> possible to compress D source by storing it in ‘tokenized’ 
>> form.
>>
>> I feel this description is unclear, and it might just reflect 
>> how DMD is implemented. I haven't implemented a C++ parser but 
>> parsers I have worked with - such as for C - it is always the 
>> case that lexer doesn't get impacted by the semantic analysis. 
>> The standard process is to get a stream of tokens from the 
>> lexer and work with that. It is also conceivable that someone 
>> could create a "dumb" AST first for C++, and as a subsequent 
>> phase add semantic meaning to the AST, just as is done for D.
>>
>> For now I propose to remove this paragraph until there is a 
>> better description available. Please would you review my pull 
>> request as it it is blocking me from doing further work.
>>
>
> Hi Walter, please would you share any insights regarding above?
>

Hi I am still waiting to hear your views on above. The pull 
request I submitted for revising the intro is stuck because of 
this.

Regards