std.d.lexer requirements

[Jonathan M Davis]

On Wednesday, August 01, 2012 22:09:05 Walter Bright wrote:
> On 8/1/2012 9:54 PM, Jonathan M Davis wrote:
> > Then just pass the same identifier table to the function which creates the
> > token range. That doesn't require another type.
> 
> You're still going to require another type, otherwise you'll have to
> duplicate the state in every token allocation, with resultant heavy memory
> and initialization costs.

Why would you be duplicating any state in the token? The range would have the 
state, not the token. The token would only have the data specific to the token, 
not the range as a whole.

- Jonathan M Davis