std.d.lexer requirements

[Jacob Carlborg]

On 2012-08-02 09:29, Walter Bright wrote:

> My experience in writing fast string based code that worked on UTF8 and
> correctly handled multibyte characters was that they are very possible
> and practical, and they are faster.
>
> The lexer MUST MUST MUST be FAST FAST FAST. Or it will not be useful. If
> it isn't fast, serious users will eschew it and will cook up their own.
> You'll have a nice, pretty, useless toy of std.d.lexer.
>
> I think there's some serious underestimation of how critical this is.

I do understand that the lexer needs to be insanely fast and it needs to 
operate on UTF-8 and not UTF-32 or anything else.

But what I still don't understand is how a UTF-8 range is going to be 
usable by other range based functions in Phobos.

-- 
/Jacob Carlborg