Request for comments: std.d.lexer

[Jonathan M Davis]

On Friday, February 08, 2013 14:29:20 Dmitry Olshansky wrote:
> If target is random access range just use offset throughout. It's
> basically becomes base + offset vs base + pointer i.e. non-issue

> If not then pointer argument no longer applies and you can just as well
> use separate counter on per popFront. It'd not that costly in this case
> and flexibility tramps other concerns with forward ranges in any case.

I don't know exactly how costly it ends up being (and certainly, poor design 
choices in other areas could dwarf that cost), but it does incur extra 
overhead throughout the lexing process. In most code, it wouldn't be a big 
deal, but the lexer is trying to be lightning fast, so every extra bit like 
that is going to add up and slow it down. But you really don't have any choice 
if you don't even have random access. Regardless, my point is that accepting 
generic ranges rather than pointers complicates things somewhat and does incur 
at least a slight performance penalty.

- Jonathan M Davis