Lexer in D

[Dmitry Olshansky]

02-Mar-2013 15:04, David пишет:
> Am 02.03.2013 10:33, schrieb Namespace:
>> For one of our courses this year we have a very cool topic: Lexer and
>> Compiler.
>> In the past I have already tried some experiments in this direction with
>> Romulus and Remus, but I've never measured my Lexing time.
>> That's why I wanted to start again to write a lexer (hitherto purely for
>> the learning effect).
>> I looked at the code of some of the other lexers, that are also written
>> in D, to get some inspiration from, but currently my Lexer is a little
>> bit slow, it requires for example ~200 msecs for std.datetime.
>> So I wanted to ask whether there are nice people who help me to improve
>> our lexer.
>>
>> Code: https://github.com/Dgame/Puzzle/blob/master/lexer.d
>>
>> Thanks in advance. :)
>
> Making `types` and `keywords` an array might improve the speed a little bit:
>
> if (id in types) -> if (types.canFind(id))
> (same with keywords)

That would be slower as canFind is linear search.
Simpler way is use first letter and length to select a bucket of keywords.

But otherwise, yes, the first advice is never use built-in AA as these 
take quite some time to allocate. Plus they not that fast to lookup as 
they used mod-prime not mod-power-of-2 to pick a bucket.


-- 
Dmitry Olshansky