Lexer in D

[Namespace]

Array has an horrible code...

I decided to start from scratch.
So I took a close look at the lexer of dmd and took over the 
basic functionality.
Result for std.datetime (without profiling so far):

------------------------
Total time (ms): 5971.66
Repetitions    : 200
Sample mode    : 28 (101 ocurrences)
Median time    : 28.89
Avg time       : 29.8583
Std dev.       : 3.37222
Minimum        : 27.958
Maximum        : 70.583
95% conf.int.  : [23.2489, 36.4677]  e = 6.60943
99% conf.int.  : [21.172, 38.5446]  e = 8.68627
EstimatedAvg95%: [29.3909, 30.3257]  e = 0.467357
EstimatedAvg99%: [29.2441, 30.4725]  e = 0.614212