Let's stop parser Hell

[Tobias Pankrath]

On Tuesday, 31 July 2012 at 16:10:14 UTC, Kai Meyer wrote:

> I know I'm a little late coming into this conversation. This 
> seems like a nice thread to toss myself into. I've started 
> working on a generic lexer that is based off of a defined 
> grammar.

Every helping hand is appreciated :-)

> As I read through the thread (I unfortunately don't have enough 
> time to read every post, but I skimmed through as many as I 
> could, and read the ones that seemed important), it seems like 
> we need a parser in D that can lex D, and provide a Range of 
> tokens that could be consumed.

Yes. To make this statement more precise: We need a lexer that 
provides
a range of tokens and we need a parser which makes it possible to 
build
an AST. Pegged combines both approaches but imposes an overhead 
if you
just need a token list. However I'm not sure if this is a problem.

There are already some working D-Parsers buried in this thread.

> With some very minor tweaks, and a properly written Grammar 
> class, I basically have it already done. D was going to be one 
> of the first languages I would have written a definition for.
>
> https://github.com/kai4785/firstfront

I've only glimpsed at your code. For most languages lexing is far 
more expensive then parsing and thus the lexer has to be very 
fast and I wouldn't pursue your approach and instead use 
something like ragel. It already has D output but needs a 
separate build step.