Lexer in D

[Dmitry Olshansky]

02-Mar-2013 23:50, Zach the Mystic пишет:
> On Saturday, 2 March 2013 at 11:59:39 UTC, Dmitry Olshansky wrote:
>> 02-Mar-2013 15:04, David пишет:
>>> Am 02.03.2013 10:33, schrieb Namespace:
>>>> For one of our courses this year we have a very cool topic: Lexer and
>>>> Compiler.
>>>> In the past I have already tried some experiments in this direction
>>>> with
>>>> Romulus and Remus, but I've never measured my Lexing time.
>>>> That's why I wanted to start again to write a lexer (hitherto purely
>>>> for
>>>> the learning effect).
>>>> I looked at the code of some of the other lexers, that are also written
>>>> in D, to get some inspiration from, but currently my Lexer is a little
>>>> bit slow, it requires for example ~200 msecs for std.datetime.
>>>> So I wanted to ask whether there are nice people who help me to improve
>>>> our lexer.
>>>>
>>>> Code: https://github.com/Dgame/Puzzle/blob/master/lexer.d
>>>>
>>>> Thanks in advance. :)
>>>
>>> Making `types` and `keywords` an array might improve the speed a
>>> little bit:
>>>
>>> if (id in types) -> if (types.canFind(id))
>>> (same with keywords)
>>
>> That would be slower as canFind is linear search.
>> Simpler way is use first letter and length to select a bucket of
>> keywords.
>>
>> But otherwise, yes, the first advice is never use built-in AA as these
>> take quite some time to allocate. Plus they not that fast to lookup as
>> they used mod-prime not mod-power-of-2 to pick a bucket.
>
> And it doesn't even need to:
> http://d.puremagic.com/issues/show_bug.cgi?id=9522

Yes, obviously AA is becoming a central pain point that needs fixing. If 
you are inclined to help I suggest you join forces with H.S. Teoh and 
Dicebot and help fixing the AA.

-- 
Dmitry Olshansky