RFC: std.json sucessor

[Brad Roberts via Digitalmars-d]

On 8/23/2014 10:46 AM, Walter Bright via Digitalmars-d wrote:
> On 8/23/2014 10:42 AM, Sönke Ludwig wrote:
>> Am 23.08.2014 19:38, schrieb Walter Bright:
>>> On 8/23/2014 9:36 AM, Sönke Ludwig wrote:
>>>> input types "string" and "immutable(ubyte)[]"
>>>
>>> Why the immutable(ubyte)[] ?
>>
>> I've adopted that basically from Andrei's module. The idea is to allow
>> processing data with arbitrary character encoding. However, the output
>> will
>> always be Unicode and JSON is defined to be encoded as Unicode, too,
>> so that
>> could probably be dropped...
>
> I feel that non-UTF encodings should be handled by adapter algorithms,
> not embedded into the JSON lexer, so yes, I'd drop that.

For performance purposes, determining encoding during lexing is useful. 
  You can avoid any conversion costs when you know that the original 
string is ascii or utf-8 or other.  The cost during lexing is 
essentially zero.  The cost of storing that state might be a concern, or 
it might be free in otherwise unused padding space.  The cost of 
re-scanning strings that can be avoided is non-trivial.

My past experience with this was in an http parser, where there's even 
more complex logic than json parsing, but the concepts still apply.