Request for comments: std.d.lexer

[FG]

On 2013-02-04 08:57, Jacob Carlborg wrote:
> On 2013-02-04 01:50, FG wrote:
>
>> Ah, fine. Then it's not that bad. :)
>> Have you already made it use slices of the whole source,
>> without copying the string values of tokens?
>> What kind of optimizations have you made?
>
> That would be interesting to hear. Three times slower than DMD doesn't sound
> good. I know that DMD is fast, but three times.

Looking at the current source, there is now a StringCache to hold the strings.
It is however also used to store all those long comments, so I believe quite
some time is unnecessarily wasted on generating hashes for them.
But probably there are other parts slowing things down.