std.d.lexer: pre-voting review / discussion

[Brian Schott]

On Wednesday, 11 September 2013 at 19:29:48 UTC, Jonathan M Davis 
wrote:
> On Wednesday, September 11, 2013 19:17:26 Manfred Nowak wrote:
>> Walter Bright wrote:
>> > Parsing D requires arbitrary lookahead.
>> 
>> Why---and since which version?
>
> IIRC, it's at least partially cause by the .. token. You have 
> to look ahead to
> figure out whether it's .. or a floating point literal. There 
> may be other cases
> as well, but that particular one was actually complained about 
> by Robert
> Schadek in his talk at dconf 2013.
>
> - Jonathan M Davis

Yeah. D requires lookahead in both lexing and parsing. Some 
examples:

* String literals such as q{}
* If statements need to determine the difference between if 
(expression) and if (type identifier = expression)
* Determining if something is a lamba expression or a function 
literal expression
* Determining if something is a declaration or a statement or an 
expression.
* Differentiating between (type).identifier and a lambda 
expression
* Differentiating between a type and an expression inside a 
typeid expression
* Differentiating between associative array key type and tuple 
slices in type suffixes