std.d.lexer requirements

[Jacob Carlborg]

On 2012-08-02 07:44, Jonathan M Davis wrote:
> On Wednesday, August 01, 2012 22:33:12 Walter Bright wrote:
>> The lexer must use char or it will not be acceptable as anything but a toy
>> for performance reasons.
>
> Avoiding decoding can be done with strings and operating on ranges of dchar,
> so you'd be operating almost entirely on ASCII. Are you saying that there's a
> performance issue aside from decoding?
>
>> Somebody has to convert the input files into dchars, and then back into
>> chars. That blows for performance. Think billions and billions of
>> characters going through, not just a few random strings.
>
> Why is there any converting to dchar going on here? I don't see why any would
> be necessary. If you reading in a file as a string or char[] (as would be
> typical), then you're operating on a string, and then the only time that any
> decoding will be necessary is when you actually need to operate on a unicode
> character, which is very rare in D's grammar. It's only when operating on
> something _other_ than a string that you'd have to actually deal with dchars.

I think you two are saying the same things.

-- 
/Jacob Carlborg