Request for comments: std.d.lexer

[dennis luehring]

Am 27.01.2013 13:31, schrieb Johannes Pfau:
> Am Sun, 27 Jan 2013 12:38:33 +0100
> schrieb Timon Gehr <timon.gehr at gmx.ch>:
>
>> On 01/27/2013 11:42 AM, Brian Schott wrote:
>> > On Sunday, 27 January 2013 at 10:17:48 UTC, Philippe Sigaud wrote:
>> >> * Having a range interface is good. Any reason why you made
>> >> byToken a class and not a struct? Most (like, 99%) of range in
>> >> Phobos are structs. Do you need reference semantics?
>> >
>> > It implements the InputRange interface from std.range so that users
>> > have a choice of using template constraints or the OO model in
>> > their code. ...
>>
>> The lexer range must be a struct.
>>
>> > ...
>> >> * A rough estimate of number of tokens/s would be good (I know
>> >> it'll vary). Walter seems to think if a lexer is not able to vomit
>> >> thousands of tokens a seconds, then it's not good. On a related
>> >> note, does your lexer have any problem with 10k+-lines files?
>> >
>> > $ time dscanner --sloc ../phobos/std/datetime.d
>> > 14950
>> >
>> > real    0m0.319s
>> > user    0m0.313s
>> > sys    0m0.006s
>> >
>> > $ time dmd -c ../phobos/std/datetime.d
>> >
>> > real    0m0.354s
>> > user    0m0.318s
>> > sys    0m0.036s
>> >
>> > Yes, I know that "time" is a terrible benchmarking tool, but they're
>> > fairly close for whatever that's worth.
>> >
>>
>> You are measuring lexing speed against compilation speed. A
>> reasonably well performing lexer is around one order of magnitude
>> faster on std.datetime. Maybe you should profile a little?
>>
>
> Profiling is always a good idea, but to be fair: His dmd was probably
> compiled with gcc and -O2 if it's a normal release build.
> So to compare that he should use gdc, -O2 -release -fno-bounds-check
> and probably more flags to compile the d code.
>

that makes no sense - we need a tiny piece of benchmark code inside of 
dmd frontend (and gdc) - these results are the only reliable/compareable 
benchmark

someone knows the place where such benchmarking can take place in the 
dmd frontend code?