Let's stop parser Hell

[Dmitry Olshansky]

On 01-Aug-12 02:54, Timon Gehr wrote:
> On 08/01/2012 12:01 AM, Philippe Sigaud wrote:
>> On Tue, Jul 31, 2012 at 11:38 PM, Dmitry Olshansky
>> <dmitry.olsh at gmail.com> wrote:
>>> Typically lexeres are regular as it allows them to be fast.
>
> Regularity of the language is not required for speed.

That's why there is "allows" (not "required") the reason in general is 
that plain deterministic automation is insanely fast compared with just 
about any CFG parsing scheme.

Yet there are certain grammars/cases where it doesn't matter much. Also 
tweaking DFA by "semantic" actions could make it handle some 
irregularities at no extra cost etc.

>
>>
>> Hmm, but then it has to treat comments a one token. So no Ddoc love?
>
> Ddoc is typically not required. By default it should be treated as
> whitespace. If it is required, one token seems reasonable: The
> post-processing of the doc comment is best done as a separate step.

Yup.

-- 
Dmitry Olshansky