std.d.lexer: pre-voting review / discussion

[deadalnix]

On Thursday, 12 September 2013 at 06:17:04 UTC, Walter Bright 
wrote:
> On 9/11/2013 10:10 PM, deadalnix wrote:
>> See my comment, it is possible, with increased parser 
>> complexity, to handle many
>> cases where you don't know what you are parsing yet. Doing so, 
>> lookahead is only
>> required to find matching closing token. I suspect that a fast 
>> path in the lexer
>> for that precise use case may be faster than buffering tokens, 
>> as it allow to
>> save one branch per token.
>
> I don't believe that, because you can see about anything for 
> tokens in lookahead and so have to duplicate nearly the whole 
> lexer anyway for the 'fast path', but you're free to try it out 
> and prove me wrong.

I plan to, but you know what it is, the best optimization is the 
one that go from non working to working state.