Request for comments: std.d.lexer

Dmitry Olshansky dmitry.olsh at gmail.com
Fri Feb 8 00:12:30 PST 2013


08-Feb-2013 12:01, Jonathan M Davis пишет:
> On Tuesday, February 05, 2013 22:51:32 Andrei Alexandrescu wrote:
>> I think it would be reasonable for a lexer to require a range of ubyte
>> as input, and carry its own decoding. In the first approximation it may
>> even require a random-access range of ubyte.
>
> Another big issue is the fact that in some ways, using a pointer like dmd's
> lexer does is actually superior to using a range. In particular, it's trivial
> to determine where in the text a token is, because you can simply subtract the
> pointer in the token from the initial pointer. Strings would be okay too,
> because you can subtract their ptr properties. But the closest that you'll get
> with ranges is to subtract their lengths, and the only ranges that are likely
> to define length are random-access ranges.

Not true, certain ranges know length but can't be random access as 
indexing is O(lgN) or worse. Including a stripe of chunks as taken from 
file.


-- 
Dmitry Olshansky


More information about the Digitalmars-d mailing list