Let's stop parser Hell

[Jonathan M Davis]

On Thursday, July 05, 2012 14:22:35 Andrei Alexandrescu wrote:
> I also am actively opposed to a project of just translating D's 
> front-end to D and dropping it into Phobos because it would smother (a) 
> work on generic parser generators, and (b) strong, dependable 
> formalization of D's syntax.

I'm not even vaguely convinced that having a lexer/parser specifically for D in 
Phobos (regardless of whether it comes from dmd or not) will have a negative 
effect on work for making generic parser generators. People are going to want a 
good parser generator _regardless_ of what the situation with parsing D is. 
And I'd be very surprised if you couldn't make a hand-written parser for D 
which was better than one that you can generate. Parser generation is great, 
because it allows you to quickly and easily put together a parser from a 
grammar, but it's unlikely to be as fast as a hand-written one optimized for a 
particular language. However, as the recent discussion on popFront shows, only 
benchmarking of actual solutions would show that for sure.

Now, the issue of a "strong, dependable formalization of D's syntax" is 
another thing entirely. Porting dmd's lexer and parser to Phobos would keep 
the Phobos implementation in line with dmd much more easily and avoid 
inconsistencies in the language definition and the like. However, if we write a 
new lexer and parser specifically for Phobos which _doesn't_ port the lexer or 
parser from dmd, then that _would_ help drive making the spec match the 
compiler (or vice versa). So, I agree that could be a definite argument for 
writing a lexer and parser from scratch rather than porting the one from dmd, 
but I don't buy the bit about it smothering parser generators at all. I think 
that the use cases are completely different.

- Jonathan M Davis