What library functionality would you most like to see in D?

[Jonathan M Davis]

> On 7/31/2011 11:32 PM, Walter Bright wrote:
> > On 7/31/2011 6:29 PM, Johann MacDonagh wrote:
> >> Basically, you give it some string (string, wstring, or dstring), and
> >> it gives
> >> you a range of tokens back. The token has the type, a slice of the
> >> input that
> >> corresponds to the token, line / column, and a value (e.g. an integer
> >> constant).
> > 
> > I suggest making the input a range.
> 
> So, the reason I didn't do this initially was to make slicing a lot
> easier. It could easily be changed to any kind of input range of
> characters. However, it looks like something that's not a direct port of
> DMD isn't worthwhile.

Some adjustment can be made. It _does_ need to have a range-based API, and 
some changes are inevitable in order to do that. But the changes to the basic 
logic need to be minimal.

The thing is that it can be an input range _and_ take advantage of slicing 
with the use of static ifs in the appropriate places. If hasSlicing!Range is 
true, then you can have a static if branch for slicing it. If it's a narrow 
string then you can use another branch. And if it's neither, then it won't get 
the advantages of slicing. It complicates the code somewhat, but the proper 
use of Phobos functions which deal with it for you should help.

In any case, the point is that we need to port dmd's lexer over to D, changing 
it enough to give it a range-based interface (so it'll take a range of dchars 
but return a range of tokens) but avoiding changing more than necessary so 
that it's easy to port changes and fixes back and forth between it and dmd's 
front end.

- Jonathan M Davis