std.d.lexer : voting thread

[Jacob Carlborg]

On 2013-10-05 02:24, Andrei Alexandrescu wrote:

> Such a trie searcher is not intelligent, but is very composable and
> extremely fast. It is just smart enough to do maximum munch (e.g.
> interprets "==" and "foreach" as one token each, not two), but is not
> smart enough to distinguish an identifier "whileTrue" from the keyword
> "while" (it claims "while" was found and stops right at the beginning of
> "True" in the stream). This is for generality so applications can define
> how identifiers work (e.g. Lisp allows "-" in identifiers but D doesn't
> etc). The trie finder doesn't do numbers or comments either. No regexen
> of any kind.

Would it be able to lex Scala and Ruby? Method names in Scala can 
contain many symbols that is not usually allowed in other languages. You 
can have a method named "==". In Ruby method names are allowed to end 
with "=", "?" or "!".

-- 
/Jacob Carlborg