std.d.lexer requirements

[Walter Bright]

On 8/3/2012 2:01 AM, Jacob Carlborg wrote:
> On 2012-08-03 09:35, Walter Bright wrote:
>
>> Look in doc.c at highlightCode2() for how to call the lexer by itself.
>
> So basically:
>
> Token tok;
>
> //start timer
>
> while (tok.value != TOKeof)
>      lex.scan(&tok);
>
> //end timer
>
> Something like that?


Pretty much. Or just call lex.nextToken() until you get a TOKeof.