std.d.lexer requirements

[Walter Bright]

On 8/2/2012 8:00 PM, Andrei Alexandrescu wrote:
> On 8/2/12 10:40 PM, Walter Bright wrote:
>> To reiterate another point, since we are in the compiler business,
>> people will expect std.d.lexer to be of top quality, not some bag on the
>> side. It needs to be usable as a base for writing a professional quality
>> compiler. It's the reason why I'm pushing much harder on this than I do
>> for other modules.
>
> The lexer must be configurable enough to tokenize other languages than D. I
> confess I'm very unhappy that there seem to be no less than three people
> determined to write lexers for D. We're wasting precious talent and resources
> doubly. Once, several people are working in parallel on the same product.
> Second, none of them is actually solving the problem that should be solved.

I agree and I hope the three can combine their efforts with the best ideas of each.

I don't think the lexer needs to be configurable to other languages. I think 
it's fine that it be custom for D. However, and this is a big however, its 
user-facing interface should be usable for other languages, I see no obvious 
reason why this cannot be.