std.d.lexer : voting thread

[Sönke Ludwig]

Yes


I see the token type discussion as a matter of taste and one that could
cleanly be changed using a deprecation path (I wouldn't mind the Tok!str
approach, though). I also see no fundamental reason why the API forbids
extension for shared sting tables or table-less lexing. And pure
implementation details (IMO) shouldn't be a primary voting concern. Much
more important is that we don't defer this another year or more for no
good reason (changing pure implementation details or purely extending
the API are no good reasons when there is a solid implementation/API
already).

(sorry for the additional rationale)