std.d.lexer : voting thread

[Andrei Alexandrescu]

On 10/9/13 6:10 PM, Martin Nowak wrote:
> On 10/08/2013 05:05 PM, Andrei Alexandrescu wrote:
>> But no matter. My most significant bit is, we need a trie lexer
>> generator ONLY from the token strings, no TK_XXX user-provided symbols
>> necessary. If all we need is one language (D) this is a non-issue
>> because the library writer provides the token definitions. If we need to
>> support user-provided languages, having the library manage the string ->
>> small integer mapping becomes essential.
>
> It's good to get rid of the symbol names.
> You should try to map the strings onto an enum so that final switch works.
>
> final switch (t.type_)
> {
> case t!"<<": break;
> // ...
> }

Excellent point! In fact one would need to use t!"<<".id instead of t!"<<".

I'll work on that next.


Andrei