std.d.lexer requirements

[Jacob Carlborg]

On 2012-08-02 22:51, Christophe Travert wrote:
> Jacob Carlborg , dans le message (digitalmars.D:174069), a écrit :
>> On 2012-08-02 10:15, Walter Bright wrote:
>>
>>> Worst case use an adapter range.
>>
>> And that is better than a plain string?
>>
> because its front method does not do any decoding.

If it was a plain string you wouldn't use "front". You would handle any, 
possible, decoding by yourself, internally in the lexer. This is what 
Johnathan already does, it seems:

static if(isNarrowString!R)
     Unqual!(ElementEncodingType!R) first = range[0];
else
     dchar first = range.front;

If you're only supporting plain UTF-8 strings you would just do:

char first = range[0];

-- 
/Jacob Carlborg