std.d.lexer: pre-voting review / discussion

deadalnix deadalnix at gmail.com
Wed Sep 11 18:30:03 PDT 2013


On Wednesday, 11 September 2013 at 23:44:55 UTC, Walter Bright 
wrote:
> On 9/11/2013 3:09 PM, Michel Fortin wrote:
>> On 2013-09-11 18:49:44 +0000, Walter Bright 
>> <newshound2 at digitalmars.com> said:
>>
>>> 6. No clue how lookahead works with this. Parsing D requires 
>>> arbitrary lookahead.
>>
>> Since this is a forward range, you can save() it at one 
>> location and continue
>> parsing, and then restart parsing from the savepoint.
>> https://github.com/Hackerpilot/phobos/blob/master/std/d/lexer.d#L2324
>>
>> This also means that the underlying range used as input to the 
>> lexer must be a
>> forward range that you can save() too.
>
> That's correct, but that implies re-lexing the tokens, which 
> has negative performance implications.

Indeed. What solution do you have in mind ?


More information about the Digitalmars-d mailing list