Request for comments: std.d.lexer

[Jonathan M Davis]

On Sunday, February 10, 2013 00:26:41 Dmitry Olshansky wrote:
> 09-Feb-2013 19:46, Andrei Alexandrescu пишет:
> > On 2/9/13 10:37 AM, Jacob Carlborg wrote:
> >> On 2013-02-09 16:10, Andrei Alexandrescu wrote:
> >>> Requiring a random-access range of ubyte with a terminating zero may be
> >>> the most general approach to a fast lexer - and that's fine.
> >> 
> >> Requiring a random-access range probably makes it easier. People here
> >> seems to try to support ranges with less functionality, like input or
> >> forward ranges.
> > 
> > Yah. The way I see it is, start with a random-access range and then see
> > what the use patterns are. Then possibly refine.
> 
> I don't get this. There is no sensible requirement to forbid non-random
> access. A couple of static ifs and you are done.
> 
> And I can confidently say that std.d.lexer has quite some room for
> optimization in both cases and it doesn't have to sacrifice the generic
> path.

It may end up being less efficient with some types of ranges, but it can still 
work, and someone's use case may not care about that extra loss of efficiency. 
Those who really do can use RA ranges or strings or whatever.

But if we throw too many extra restrictions on the ranges (like they have to 
be random access or end with 0), then pretty soon you might as well require 
that they use string, because the extra benefit of the few extra types of 
ranges which would work wolud be pretty minimal.

- Jonathan M Davis