parseJSON bug

[Nick Sabalausky]

On Thu, 08 Aug 2013 17:17:38 +0200
"Tofu Ninja" <emmons0 at purdue.edu> wrote:

> On Thursday, 8 August 2013 at 13:56:15 UTC, Dicebot wrote:
> > On Thursday, 8 August 2013 at 13:49:22 UTC, bearophile wrote:
> >> In my opinion we should follow the formal JSON grammar.
> >
> > This. Anyone who wants JavaScript behavior can use own 
> > third-party library bust standard library must behave according 
> > to published standards and specifications.
> 
> A formal grammar can be found here starting on page 202 but I 
> don't know enough about grammars to be able to interpret it.
> http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-262.pdf
> 
> Im starting to become less sure if its a bug or not...

I think that makes it clear that whitespace is NOT allowed within
numeric or fractional literals. Note that the JSONNumber and
JSONFraction tokens are defined in the lexical section (15.12.1.1). In
a lexer, whitespace is something that is always handled explicitly (as
they do, in both the JSONWhiteSpace and JSONStringCharacter tokens).

Down in the "Syntactic Grammar" section (15.12.1.2), they don't
explicitly mention anything about whitespace (for example, in the
JSONElementList token, where we already *know* whitespace is allowed).
This is undoubtedly because (aside from lexing) whitespace is generally
NOT explicit, but rather implied to optionally exist before/after each
token.

It sounds like the bug is that std.json is currently treating JSONNumber
(and maybe JSONFraction too) as a parser-handled nonterminal, instead
of a lexer-handled terminal as the spec indicates.