std.d.lexer requirements

[Jonathan M Davis]

On Thursday, August 02, 2012 10:58:21 Walter Bright wrote:
> On 8/2/2012 4:49 AM, deadalnix wrote:
> > How is that different than a manually done range of dchar ?
> 
> The decoding is rarely necessary, even if non-ascii data is there. However,
> the range cannot decide if decoding is necessary - the receiver has to,
> hence the receiver does the decoding.

Hence why special-casing must be used to deal with variably-length encodings 
like UTF-8. If it's not something that the range can handle through front and 
popFront, then it's not going to work as a range. Having it be _possible_ to 
special-case but not require it allows for the type to works a range but still 
be efficient of the function operating on the range codes for it. But if you 
require the function to code for it, then it's not really a range.

- Jonathan M Davis