Request for comments: std.d.lexer

[Jonathan M Davis]

On Friday, February 08, 2013 23:48:50 Walter Bright wrote:
> On 2/8/2013 12:01 AM, Jonathan M Davis wrote:
> > It's not quite a use case where
> > ranges shine - especially when efficiency is a top priority.
> 
> A more problematic case is dmd's lexer relies on a 0 byte at the end to be a
> "sentinel" for the end of file. Without such a sentinel, every consumption
> of a source character requires two operations rather than one.

I didn't know that. That's a cute trick. But yeah, without controlling the 
input, it's not possible, and that won't work with a general implementation in 
a library. It would be trivial enough to put a wrapper around the input to add 
the 0 byte at the end, but the wrapper would still have to keep checking for 
empty, so you wouldn't gain anything.

- Jonathan M Davis