Request for comments: std.d.lexer

[Walter Bright]

On 1/27/2013 12:11 PM, Dmitry Olshansky wrote:
> 27-Jan-2013 23:46, Walter Bright пишет:
>> On 1/27/2013 1:51 AM, Brian Schott wrote:
>>> I'm interested in ideas on the API design and other high-level issues
>>> at the
>>> moment. I don't consider this ready for inclusion. (The current module
>>> being
>>> reviewed for inclusion in Phobos is the new std.uni.)
>>
>> Just a quick comment: byToken() should not accept a filename. It's input
>> should be via an InputRange, not a file.
>
> InputRange of char/wchar/dchar I guess as lexer should have an ability to avoid
> decoding.

Just char would suffice. If a user needs wchar/dchar, they can put a map in 
between the InputRange and the lexer.