RFC: std.json sucessor

[Don via Digitalmars-d]

On Monday, 25 August 2014 at 14:04:12 UTC, Sönke Ludwig wrote:
> Am 25.08.2014 15:07, schrieb Don:
>> On Thursday, 21 August 2014 at 22:35:18 UTC, Sönke Ludwig 
>> wrote:
>>> Following up on the recent "std.jgrandson" thread [1], I've 
>>> picked up
>>> the work (a lot earlier than anticipated) and finished a 
>>> first version
>>> of a loose blend of said std.jgrandson, vibe.data.json and 
>>> some
>>> changes that I had planned for vibe.data.json for a while. 
>>> I'm quite
>>> pleased by the results so far, although without a 
>>> serialization
>>> framework it still misses a very important building block.
>>>
>>> Code: https://github.com/s-ludwig/std_data_json
>>> Docs: http://s-ludwig.github.io/std_data_json/
>>> DUB: http://code.dlang.org/packages/std_data_json
>>>
>>> The new code contains:
>>> - Lazy lexer in the form of a token input range (using slices 
>>> of the
>>>   input if possible)
>>> - Lazy streaming parser (StAX style) in the form of a node 
>>> input range
>>> - Eager DOM style parser returning a JSONValue
>>> - Range based JSON string generator taking either a token 
>>> range, a
>>>   node range, or a JSONValue
>>> - Opt-out location tracking (line/column) for tokens, nodes 
>>> and values
>>> - No opDispatch() for JSONValue - this has shown to do more 
>>> harm than
>>>   good in vibe.data.json
>>>
>>> The DOM style JSONValue type is based on 
>>> std.variant.Algebraic. This
>>> currently has a few usability issues that can be solved by
>>> upgrading/fixing Algebraic:
>>>
>>> - Operator overloading only works sporadically
>>> - No "tag" enum is supported, so that switch()ing on the type 
>>> of a
>>>   value doesn't work and an if-else cascade is required
>>> - Operations and conversions between different Algebraic 
>>> types is not
>>>   conveniently supported, which gets important when other 
>>> similar
>>>   formats get supported (e.g. BSON)
>>>
>>> Assuming that those points are solved, I'd like to get some 
>>> early
>>> feedback before going for an official review. One open issue 
>>> is how to
>>> handle unescaping of string literals. Currently it always 
>>> unescapes
>>> immediately, which is more efficient for general input ranges 
>>> when the
>>> unescaped result is needed, but less efficient for string 
>>> inputs when
>>> the unescaped result is not needed. Maybe a flag could be 
>>> used to
>>> conditionally switch behavior depending on the input range 
>>> type.
>>>
>>> Destroy away! ;)
>>>
>>> [1]: 
>>> http://forum.dlang.org/thread/lrknjl$co7$1@digitalmars.com
>>
>>
>> One missing feature (which is also missing from the existing 
>> std.json)
>> is support for NaN and Infinity as JSON values. Although they 
>> are not
>> part of the formal JSON spec (which is a ridiculous omission, 
>> the
>> argument given for excluding them is fallacious), they do get 
>> generated
>> if you use Javascript's toString to create the JSON. Many JSON 
>> libraries
>> (eg Google's) also generate them, so they are frequently 
>> encountered in
>> practice. So a JSON parser should at least be able to lex them.
>>
>> ie this should be parsable:
>>
>> {"foo": NaN, "bar": Infinity, "baz": -Infinity}
>
> This would probably best added as another (CT) optional 
> feature. I think the default should strictly adhere to the JSON 
> specification, though.

Yes, it should be optional, but not a compile-time option.
I think it should parse it, and based on a runtime flag, throw an 
error (perhaps an OutOfRange error or something, and use the same 
thing for values that exceed the representable range).

An app may accept these non-standard values under certain 
circumstances and not others. In real-world code, you see a *lot* 
of these guys.

Part of the reason these are important, is that NaN or Infinity 
generally means some Javascript code just has an uninitialized 
variable. Any other kind of invalid JSON typically means 
something very nasty has happened. It's important to distinguish 
these.