What library functionality would you most like to see in D?

[Johann MacDonagh]

On 8/1/2011 5:50 PM, Jonathan M Davis wrote:
>> On 7/31/2011 11:32 PM, Walter Bright wrote:
>>> On 7/31/2011 6:29 PM, Johann MacDonagh wrote:
>>>> Basically, you give it some string (string, wstring, or dstring), and
>>>> it gives
>>>> you a range of tokens back. The token has the type, a slice of the
>>>> input that
>>>> corresponds to the token, line / column, and a value (e.g. an integer
>>>> constant).
>>>
>>> I suggest making the input a range.
>>
>> So, the reason I didn't do this initially was to make slicing a lot
>> easier. It could easily be changed to any kind of input range of
>> characters. However, it looks like something that's not a direct port of
>> DMD isn't worthwhile.
>
> Some adjustment can be made. It _does_ need to have a range-based API, and
> some changes are inevitable in order to do that. But the changes to the basic
> logic need to be minimal.
>
> The thing is that it can be an input range _and_ take advantage of slicing
> with the use of static ifs in the appropriate places. If hasSlicing!Range is
> true, then you can have a static if branch for slicing it. If it's a narrow
> string then you can use another branch. And if it's neither, then it won't get
> the advantages of slicing. It complicates the code somewhat, but the proper
> use of Phobos functions which deal with it for you should help.
>
> In any case, the point is that we need to port dmd's lexer over to D, changing
> it enough to give it a range-based interface (so it'll take a range of dchars
> but return a range of tokens) but avoiding changing more than necessary so
> that it's easy to port changes and fixes back and forth between it and dmd's
> front end.
>
> - Jonathan M Davis

Ok, I'll work on that this weekend. I'll do as close to a port as 
possible while stripping out anything that doesn't make sense and 
D-ifying the code a little bit. It will accept anything that 
isInputRange!R && isSomeChar!(ElementEncodingType!R) and return an input 
range of tokens. I've already looked through quite a bit of the lexer.c 
source while writing my other lexer, and the port should be easy (famous 
last words).