std.data.json formal review

[Sönke Ludwig via Digitalmars-d]

Am 30.07.2015 um 05:25 schrieb Walter Bright:
> On 7/29/2015 1:41 PM, Sönke Ludwig wrote:
>> The token level is useful for reasoning about the text representation.
>> It could
>> be used for example to implement syntax highlighting, or for using the
>> location
>> information to mark errors in the source code.
>
> Ok, I see your point. The question then becomes does the node stream
> really add enough value to justify its existence, as it greatly overlaps
> the token stream.
>

I agree that in case of JSON their difference can be a bit subtle. 
Basically the node stream adds knowledge about the nesting of elements, 
as well as adding semantic meaning to special token sequences that the 
library users would otherwise have to parse themselves. Finally, it also 
guarantees a valid JSON structure, while a token range could have tokens 
in any order.

Especially the knowledge about nesting is also a requirement for the 
high-level pull parser functions (skipToKey, readArray, readString etc.) 
that make working with that kind of pull parser interface actually 
bearable, outside of mechanic code like a generic serialization framework.