Proposal for SentinelInputRange

[Andrei Alexandrescu]

On 2/28/13 12:20 PM, Dmitry Olshansky wrote:
> 28-Feb-2013 19:31, Andrei Alexandrescu пишет:
>> On 2/28/13 5:54 AM, Walter Bright wrote:
>>> On 2/27/2013 11:55 PM, Jonathan M Davis wrote:
>>>>> Again, please see how lexer.c works. I assure you, there is no double
>>>>> copying going on, nor is there a double test for the terminating 0.
>>>>
>>>> I know what the lexer does, and remember that it _doesn't_ operate on
>>>> ranges,
>>>> and there are subtle differences between being able to just use char*
>>>> and
>>>> trying to handle generic ranges.
>>>
>>> Hence the need to invent SentinelInputRange.
>>
>> I don't think the sentinel input range is a blocker for redoing the
>> parser (with ranges) in D. This discussion has probably run its course.
>
>
>> The right thing to do at this point is port the lexer and figure what
>> primitives are necessary from its input.
>>
>
> No need to port - look at std.d.lexer again. It was revamped and is
> ready for the new round of review I should say. Let's not use the old
> source for the new module and go the long path of :
> split off --> port --> patch up --> D-ify & re-write to ranges
>
> Instead we can just tweak the current std.d.lexer a little bit more and
> we have good clean-room lexer written in the idiomatic D. Well, it's
> getting there w.r.t. idiomaticness but it supports ranges including both
> random-access and forward ones (by transparently specializing for each
> one).

I think that's a good idea but I took a look at 
https://github.com/bhelyer/std.d.lexer/blob/master/std/d/lexer.d and I 
will destroy it. In a good sense :o).

Andrei