[spec] Phases of translation

[Dibyendu Majumdar]

On Monday, 20 May 2019 at 16:13:39 UTC, Walter Bright wrote:

> The crucial thing to know is that the tokenizing is independent 
> of parsing, and parsing is independent of semantic analysis.
>

I am trying to understand this aspect - I found the small write 
up in the Intro section not very clear. Would be great if you 
could so a session on how things work - maybe video cast?

For example, the mixin declaration has to convert a string to AST 
I guess? When does this happen? Does it not need to invoke the 
lexer on the generated string and build AST while already in the 
semantic stage?

> I.e. the definition of a token does not change depending on 
> what construct is being parsed, and the AST generated by the 
> parser can be created without doing any semantic analysis 
> (unlike C++).
>
> These consequences fall out of the rest of the spec, hence they 
> should be more of a clarification in the introduction. The idea 
> is to head off attempts to add changes to D that introduce 
> dependencies. Such proposals do crop up from time to time, for 
> example, user-defined operator tokens.

I agree - hence I think we need to be explicit about what D 
requires of each phase. That way any change to the language can 
be subjected to a test - does it break some of the fundamental 
requirements for parsing or semantic analysis etc.

Thanks and Regards
Dibyendu