[Issue 949] Wrong spec/compiler behaviour for Strings, Integers and Floats

[d-bugmail at puremagic.com]

http://d.puremagic.com/issues/show_bug.cgi?id=949


hennrich.bloebaum at gmx.de changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|Wrong spec/compiler         |Wrong spec/compiler
                   |behaviour for DecimalFloat  |behaviour for Strings,
                   |and HexFloat                |Integers and Floats




------- Comment #1 from hennrich.bloebaum at gmx.de  2007-02-12 16:18 -------
Now, i had a closer look at the grammar for strings, integers and floats.

- I think escape sequence \<eof> should give an error instead of a string "\\".
If no error is desired a string "\0" seems to match better.
- Octals in escape sequences are lexed incorrect: spec says that \0_7 should be
the same as \07 but it's lexed as \0 and _7. I belive the spec is wrong.
- Same thing with hexadecimal escape sequences \x \u \U: spec says with
underscores, but lexed without.
- Hex strings can contain underscores according to spec, but lexed without.

I rewrote the grammar a little -g- bit to fix all issues with these
underscores:

- US in hex escape sequences are not allowed, lexer fits.
- No single . is allowed in HexFloat, lexer fails.
- More than one 0 between . and nonzeros is allowed, lexer fits.
- No single US are allowed, lexer fails sometimes:
- DecimalFloats starting with US are disallowed, lexer fits.
- ._1 is disallowed, lexer fits.
- 1._1 is allowed, lexer fits.
- 1._ is disallowed, lexer fails.
- exponents with only US are not allowed, lexer failes on exp starting with US.

Found nothing more, yet. Hope this is not too much. Comments?


##### Integer literals #####

IntegerLiteral:
        Integer
        Integer IntegerSuffix

Integer:
        Decimal
        Binary
        Octal
        Hexadecimal
        Integer

IntegerSuffix:
        L
        u
        U
        Lu
        LU
        uL
        UL

Decimal:
        0
        NonZeroDigit
-       NonZeroDigit DecimalDigits
+       NonZeroDigit DecimalDigitsUS

Binary:
-       0b BinaryDigits
-       0B BinaryDigits
+       BinPrefix BinaryDigitsNoSingleUS

+BinPrefix:
+       0b
+       0B

Octal:
-       0 OctalDigits
+       0 OctalDigitsUS

Hexadecimal:
-       0x HexDigits
-       0X HexDigits
+       HexPrefix HexDigitsNoSingleUS

NonZeroDigit:
        1
        2
        3
        4
        5
        6
        7
        8
        9

DecimalDigits:
        DecimalDigit
        DecimalDigit DecimalDigits

+DecimalDigitsUS:
+       DecimalDigitUS
+       DecimalDigitUS DecimalDigitsUS

+DecimalDigitsNoSingleUS:
+       DecimalDigit
+       DecimalDigit DecimalDigitsUS
+       DecimalDigitsUS DecimalDigit

+DecimalDigitsNoStartingUS:
+       DecimalDigit
+       DecimalDigit DecimalDigitsUS

DecimalDigit:
        0
        NonZeroDigit
-       _

+DecimalDigitUS:
+       DecimalDigit
+       _

-BinaryDigits:
-       BinaryDigit
-       BinaryDigit BinaryDigits

+BinaryDigitsUS:
+       BinaryDigitUS
+       BinaryDigitUS BinaryDigitsUS

+BinaryDigitsNoSingleUS:
+       BinaryDigit
+       BinaryDigit BinaryDigitsUS
+       BinaryDigitsUS BinaryDigit

BinaryDigit:
        0
        1
-       _

+BinaryDigitUS:
+       BinaryDigit
+       _

OctalDigits:
        OctalDigit
        OctalDigit OctalDigits

+OctalDigitsUS:
+       OctalDigitUS
+       OctalDigitUS OctalDigitsUS

OctalDigit:
        0
        1
        2
        3
        4
        5
        6
        7
-       _

+OctalDigitUS:
+       OctalDigit
+       _

HexDigits:
        HexDigit
        HexDigit HexDigits

+HexDigitsUS:
+       HexDigitUS
+       HexDigitUS HexDigitsUS

+HexDigitsNoSingleUS:
+       HexDigit
+       HexDigit HexDigitsUS
+       HexDigitsUS HexDigit

HexDigit:
        DecimalDigit
-       a
-       b
-       c
-       d
-       e
-       f
-       A
-       B
-       C
-       D
-       E
-       F
+       HexLetter

+HexDigitUS:
+       DecimalDigitUS
+       HexLetter

+HexLetter:
+       a
+       b
+       c
+       d
+       e
+       f
+       A
+       B
+       C
+       D
+       E
+       F

##### Floating literals #####

FloatLiteral:
        Float
        Float FloatSuffix
        Float ImaginarySuffix
        Float FloatSuffix ImaginarySuffix

Float:
        DecimalFloat
        HexFloat

DecimalFloat:
-       DecimalDigits .
-       DecimalDigits . DecimalDigits
-       DecimalDigits . DecimalDigits DecimalExponent
-       . Decimal
-       . Decimal DecimalExponent
-       DecimalDigits DecimalExponent
+       DecimalDigitsNoStartingUS .
+       DecimalDigitsNoStartingUS . DecimalDigitsNoSingleUS
+       DecimalDigitsNoStartingUS . DecimalDigitsNoSingleUS DecimalExponent
+       . DecimalDigitsNoStartingUS
+       . DecimalDigitsNoStartingUS DecimalExponent
+       DecimalDigitsNoStartingUS DecimalExponent

DecimalExponent:
-       e DecimalDigits
-       E DecimalDigits
-       e+ DecimalDigits
-       E+ DecimalDigits
-       e- DecimalDigits
-       E- DecimalDigits
+       DecimalExponentStart DecimalDigitsNoSingleUS

+DecimalExponentStart:
+       e
+       E
+       e+
+       E+
+       e-
+       E-

HexFloat:
-       HexPrefix HexDigits . HexDigits HexExponent
-       HexPrefix . HexDigits HexExponent
-       HexPrefix HexDigits HexExponent
+       HexPrefix HexDigitsNoSingleUS . HexDigitsNoSingleUS HexExponent
+       HexPrefix . HexDigitsNoSingleUS HexExponent
+       HexPrefix HexDigitsNoSingleUS . HexExponent
+       HexPrefix HexDigitsNoSingleUS HexExponent

HexPrefix:
        0x
        0X

HexExponent:
-       p DecimalDigits
-       P DecimalDigits
-       p+ DecimalDigits
-       P+ DecimalDigits
-       p- DecimalDigits
-       P- DecimalDigits
+       HexExponentStart DecimalDigitsNoSingleUS

+HexExponentStart:
+       p
+       P
+       p+
+       P+
+       p-
+       P-

FloatSuffix:
        f
        F
        L

ImaginarySuffix:
        i


--