Request for comments: std.d.lexer

Philippe Sigaud philippe.sigaud at gmail.com
Sun Jan 27 12:45:13 PST 2013


>> ... which is exactly what parsing expression grammars (and other
>> scannerless parsers) do.
>>
>
> That is only a happenstance and is highly overrated. What does it buy you is
> the right question to ask. After all every LL-parser can be written avoiding
> notion of lexer and strangely I see no fuss about it.
> So is it just hype? I dunno.

I don't know. I discovered what scannerless parsing was after
discovering (and getting interested) in PEG.
That's a somewhat unusual path, as I'm now discovering standard
lexer+parser parsing :)

Anyway, let's not derail the OP thread too much.

>> AFAICT, one interesting consequence is the ability to have composition
>> of grammars, which I sure have with Pegged. But maybe it's not related
>> that much, that's not something I stopped to think about.
>> In any case, grammar composition is something I've learnt to like quite a
>> lot.
>>
>
> You can still have composability of grammars. In fact I'd define a couple of
> a less powerful but better optimized variations if I were you.

Yes, getting an automaton to deal with the regular part would be good.
But then, there already is std.regex ;)

it's good to know composition is possible elsewhere. I really like it:
it makes me use grammars (and the related, generated parsers) as I use
functions in PL: slowly building layers of abstraction and reuse.

> PEG-style memoization is still possible as long as underlying grammars are
> context-free all the useful usually are.

Yes, I agree.


> LR hardly composable ( I haven't seen a single one that does).
> But all top-down one always are unless there is some explicit work spent to
> undermine this ability ;)

What I still can't get an easy answer on is: is the D grammar LR(1),
LR(0), LALR?


More information about the Digitalmars-d mailing list