Official D Grammar

[Artur Skawina]

On 04/07/13 00:35, Bruno Medeiros wrote:
> On 06/04/2013 20:52, Artur Skawina wrote:
>> On 04/06/13 17:21, Bruno Medeiros wrote:
>>> On 02/04/2013 00:18, Brian Schott wrote:
>>>> I've pretty much finished up my work on the std.d.lexer module. I am
>>>> waiting for the review queue to make some progress on the other (three?)
>>>> modules being reviewed before starting a thread on it.
>>>>
>>>
>>> BTW, even in the lexer spec I've found an issue. How does this parse:
>>>    5.blah
>>> According to the spec (maximal munch technique), it should be FLOAT then IDENTIFIER. But DMD parses it as INTEGER DOT IDENTIFIER. I'm assuming the lastest is the correct behavior, so you can write stuff like 123.init, but that should be clarified.
>>
>> "1..2", "1.ident" and a float literal with '_' after the '.' are the
>> DecimalFloat cases that I immediately ran into when doing a lexer based on
>> the dlang grammar. It's obvious to a human how these should be handled, but
>> code generators aren't that smart... But they are good at catching mistakes
>> like these.
> 
> The "1..2" is actually mentioned in the spec:
> "An exception to this rule is that a .. embedded inside what looks like two floating point literals, as in 1..2, is interpreted as if the .. was separated by a space from the first integer."
> so it's there, even if it can be missed.

I know, but documenting a (grammar) bug does not make it go away.

> But unless I missed it, the spec is incorrect for the "1.ident" or "1_2_3_4_5_6_._5_6_7_8" cases as there is no exception mentioned there... and it's not always 100% obvious to a human how these should be handled. Or maybe that's just me :)

What does the "spec" currently say about ".001"?..

It's been a while since I did a d lexer, based on the dlang grammar - it 
(the lexer) was supposed to be dmd compatible. Took a closer look at the
actual dlang.org rules today while writing this message...


Will try to find some time to clean up and convert a working D lexical
grammar to PEG; what i have should be 1:1 translatable, except one rule
(DelimitedString) and put it on the wiki. Maybe it will help someone
avoid these issues.

artur