RFC: std.json sucessor

[Sönke Ludwig via Digitalmars-d]

Am 22.08.2014 17:47, schrieb Jacob Carlborg:
> On 2014-08-22 00:35, Sönke Ludwig wrote:
>> Following up on the recent "std.jgrandson" thread [1], I've picked up
>> the work (a lot earlier than anticipated) and finished a first version
>> of a loose blend of said std.jgrandson, vibe.data.json and some changes
>> that I had planned for vibe.data.json for a while. I'm quite pleased by
>> the results so far, although without a serialization framework it still
>> misses a very important building block.
>>
>> Code: https://github.com/s-ludwig/std_data_json
>> Docs: http://s-ludwig.github.io/std_data_json/
>> DUB: http://code.dlang.org/packages/std_data_json
>
> * Opening braces should be put on their own line to follow Phobos style
> guides

Will do.

> * I'm wondering about the assert in lexer.d, line 160. What happens if
> two invalid tokens after each other occur?

There are actually no invalid tokens at all, the "invalid" enum value is 
only used to denote that no token is currently stored in _front. If 
readToken() doesn't throw, there will always be a valid token.

> * I think we have talked about this before, when reviewing D lexers. I'm
> thinking of how to handle invalid data. Is it the best solution to throw
> an exception? Would it be possible to return an error token and have the
> client decide what to do about? Shouldn't it be possible to build a JSON
> validator on this?

That would indeed be a possibility, it's how I used to handle it in my 
private version of std.lexer, too. It could also be made a compile time 
option.

> * The lexer seems to always convert JSON types to their native D types,
> is that wise to do? That's unnecessary if you're implementing syntax
> highlighting

It's basically the same trade-off as for unescaping string literals. For 
"string" inputs, it would be more efficient to just store a slice, but 
for generic input ranges it avoids the otherwise needed allocation. The 
proposed flag could make an improvement here, too.