std.d.lexer: pre-voting review / discussion

[H. S. Teoh]

On Thu, Sep 12, 2013 at 04:21:36AM +0200, Martin Nowak wrote:
> On 09/12/2013 03:39 AM, Walter Bright wrote:
> >On 9/11/2013 6:30 PM, deadalnix wrote:
> >>Indeed. What solution do you have in mind ?
> >
> >The solution dmd uses is to put in an intermediary layer that saves
> >the lookahead tokens in a linked list.
> 
> Linked list sounds bad.
> Do you have a rough idea how often lookahead is needed, i.e. is it
> performance relevant? If so it might be worth tuning.

I still don't understand why backtracking is necessary in the first
place. I would've thought a modern parser should be well able to encode
seen tokens in its state so that backtracking is never necessary. Or
does D grammar have tricky bits that cannot be handled this way, that
I'm unaware of?


T

-- 
The peace of mind---from knowing that viruses which exploit Microsoft
system vulnerabilities cannot touch Linux---is priceless. -- Frustrated
system administrator.