std.d.lexer requirements

[Walter Bright]

On 8/1/2012 9:06 PM, Jonathan M Davis wrote:
> On Wednesday, August 01, 2012 21:00:53 Walter Bright wrote:
>> On 8/1/2012 7:04 PM, deadalnix wrote:
>>> Le 02/08/2012 02:10, Walter Bright a écrit :
>>>> 6. A single "Lexer" instance should be able to serially accept input
>>>> ranges, sharing and updating one identifier table
>>>
>>> I see the lexer as a function that take an range of char as input and give
>>> back a range of token. Does it make sense to make an instance of a lexer
>>> ?
>> Yes, as the lexer will have some state and some configuration parameters.
>
> Couldn't that just be part of the range type? A separate lexer type shouldn't
> be necessary.

No, because the user may want to serially present ranges to the same lexer type. 
Why do that, you might say? That's how dmd works - it means the identifier table 
will be the same for all source files.