RFC: std.json sucessor

[Don via Digitalmars-d]

On Thursday, 21 August 2014 at 22:35:18 UTC, Sönke Ludwig wrote:
> Following up on the recent "std.jgrandson" thread [1], I've 
> picked up the work (a lot earlier than anticipated) and 
> finished a first version of a loose blend of said 
> std.jgrandson, vibe.data.json and some changes that I had 
> planned for vibe.data.json for a while. I'm quite pleased by 
> the results so far, although without a serialization framework 
> it still misses a very important building block.
>
> Code: https://github.com/s-ludwig/std_data_json
> Docs: http://s-ludwig.github.io/std_data_json/
> DUB: http://code.dlang.org/packages/std_data_json
>
> The new code contains:
>  - Lazy lexer in the form of a token input range (using slices 
> of the
>    input if possible)
>  - Lazy streaming parser (StAX style) in the form of a node 
> input range
>  - Eager DOM style parser returning a JSONValue
>  - Range based JSON string generator taking either a token 
> range, a
>    node range, or a JSONValue
>  - Opt-out location tracking (line/column) for tokens, nodes 
> and values
>  - No opDispatch() for JSONValue - this has shown to do more 
> harm than
>    good in vibe.data.json
>
> The DOM style JSONValue type is based on std.variant.Algebraic. 
> This currently has a few usability issues that can be solved by 
> upgrading/fixing Algebraic:
>
>  - Operator overloading only works sporadically
>  - No "tag" enum is supported, so that switch()ing on the type 
> of a
>    value doesn't work and an if-else cascade is required
>  - Operations and conversions between different Algebraic types 
> is not
>    conveniently supported, which gets important when other 
> similar
>    formats get supported (e.g. BSON)
>
> Assuming that those points are solved, I'd like to get some 
> early feedback before going for an official review. One open 
> issue is how to handle unescaping of string literals. Currently 
> it always unescapes immediately, which is more efficient for 
> general input ranges when the unescaped result is needed, but 
> less efficient for string inputs when the unescaped result is 
> not needed. Maybe a flag could be used to conditionally switch 
> behavior depending on the input range type.
>
> Destroy away! ;)
>
> [1]: http://forum.dlang.org/thread/lrknjl$co7$1@digitalmars.com


One missing feature (which is also missing from the existing 
std.json) is support for NaN and Infinity as JSON values. 
Although they are not part of the formal JSON spec (which is a 
ridiculous omission, the argument given for excluding them is 
fallacious), they do get generated if you use Javascript's 
toString to create the JSON. Many JSON libraries (eg Google's) 
also generate them, so they are frequently encountered in 
practice. So a JSON parser should at least be able to lex them.

ie this should be parsable:

{"foo": NaN, "bar": Infinity, "baz": -Infinity}

You should also put tests in for what happens when you pass NaN 
or infinity to toJSON. It shouldn't silently generate invalid 
JSON.