std.d.lexer : voting thread

[Martin Nowak]

On 10/06/2013 10:18 AM, Sönke Ludwig wrote:
> I also see no fundamental reason why the API forbids
> extension for shared sting tables or table-less lexing.

The current API requires to copy slices of the const(ubyte)[] input to 
string values in every token. This can't be done efficiently without a
string table. But a string table is unnecessary for many use-cases,
so the API has a built-in performance/memory issue.