Looking for champion - std.lang.d.lex

Jacob Carlborg doob at me.com
Tue Oct 26 00:53:48 PDT 2010


On 2010-10-26 04:44, Nick Sabalausky wrote:
> "Walter Bright"<newshound2 at digitalmars.com>  wrote in message
> news:ia59si$1r0j$1 at digitalmars.com...
>>
>> Consider a string literal, say "abc\"def". With Goldie's method, I infer
>> this string has to be scanned twice. Once to find its limits, and the
>> second to convert it to the actual string.
>
> Yea, that is true. With that string in the input, the value given to the
> user code will be:
>
> assert(tokenObtainedFromGoldie.toString() == q{"abc\"def"});
>
> That's a consequence of the grammar being separated from lexing/parsing
> implementation.
>
> You're right that that does seem less than ideal. Although I'm not sure how
> to remedy that without loosing the independence between grammar and
> lex/parse implementation that is the main point of the GOLD-based style.
>
> But there's something I don't quite understand about the approach you're
> suggesting: You seem to be suggesting that a terminal be progressively
> converted into its final form *as* it's still in the process of being
> recognized by the DFA. Which means, you don't know *what* you're supposed to
> be converting it into *while* you're converting it.

I don't have much knowledge in this area but isn't this what a 
look-ahead is for? Just look ahead (hopefully) one character and decide 
what to convert to.

-- 
/Jacob Carlborg


More information about the Digitalmars-d mailing list