std.d.lexer: pre-voting review / discussion

[deadalnix]

On Thursday, 12 September 2013 at 14:09:43 UTC, H. S. Teoh wrote:
> This can be handled by using an intermediate grammar rule. 
> Reduce all
> (...) into an intermediate type, say ArgList, so the reduction
> happens something like this:
>
> 	int  foo   ()      ()      {}
> 	Type Ident ArgList ArgList ^
>
> Then have the rule:
>
> 	CompileTimeArgs ::= ArgList
> 	RuntimeArgs ::= ArgList
> 	TemplateDecl ::= Type Ident CompileTimeArgs RuntimeArgs '{' ...
> 	FuncDecl ::= Type Ident RuntimeArgs '{' ...
>
> So first, all (...) gets parsed to ArgList, but it's not yet 
> fixed
> whether they are compile-time arguments or runtime arguments. 
> It's only
> after you see the next '(' or '{' that you decide whether 
> ArgList should
> reduce to CompileTimeArgs or RuntimeArgs.
>
> ArgList itself, of course, will accept all possible parameters 
> (both
> runtime and compile-time): types, expressions, symbols. Then 
> when you
> reduce it to RuntimeArgs, you reject stuff that can't be 
> interpreted as
> parameter declarations.
>

And then you got to backtrack the parsing instead of the lexing. 
You just moved the problem around. You'll have to create some 
temporary ast nodes that then will fix into what they really are.