std.d.lexer: pre-voting review / discussion

[Walter Bright]

On 9/11/2013 11:45 AM, qznc wrote:
> On Wednesday, 11 September 2013 at 15:02:00 UTC, Dicebot wrote:
>> std.d.lexer is standard module for lexing D code, written by Brian Schott
>>
>> Documentation:
>> http://hackerpilot.github.io/experimental/std_lexer/phobos/lexer.html
>
> The documentation for Token twice says "measured in ASCII characters or UTF-8
> code units", which sounds confusing to me.
>
> Is it UTF-8, which includes ASCII? Then it should not be "or".

Pedantically, it is just UTF-8 code units, which are a superset of ASCII.