std.d.lexer performance (WAS: std.d.lexer : voting thread)

[Dejan Lekic]

On Thursday, 3 October 2013 at 20:11:02 UTC, Andrei Alexandrescu 
wrote:
> On 10/3/13 12:47 PM, Brian Schott wrote:
>> On Thursday, 3 October 2013 at 19:07:03 UTC, nazriel wrote:
>>> (Btw, someone got benchmarks of std.d.lexer?
>>> I remember that Brain was benchmarking his module quite a lot 
>>> in order
>>> to catch up with DMD's lexer but I can't find links in IRC 
>>> logs. I
>>> wonder if he achieved his goal in this regard)
>>
>> The most recent set of timings that I have can be found here:
>> https://raw.github.com/Hackerpilot/hackerpilot.github.com/master/experimental/std_lexer/images/times4.png
>>
>>
>> They're a bit old at this point, but not much has changed in 
>> the lexer
>> internals. I can try running another set of benchmarks soon. 
>> (The
>> hardest part is hacking DMD to just do the lexing)
>>
>> The times on the X-axis are milliseconds.
>
> I see we're considerably behind dmd. If improving performance 
> would come at the price of changing the API, it may be sensible 
> to hold off adoption for a bit.
>
> Andrei

Quite frankly, I (or better say many of us) need a COMPLETE D 
lexer that is UP TO DATE. std.lexer should be, if it is a Phobos 
module, and that is all that matters. Performance optimizations 
can come later.

So what if it's API will change? We, who use D2 since the very 
beginning, are used to it! API changes can be done smoothly, with 
phase-out stages. People would be informed what pieces of the API 
will become deprecated, and it is their responsibility to fix 
their code to reflect such changes. All that is needed is little 
bit of planning...