std.d.lexer requirements

[Christophe Travert]

Jonathan M Davis , dans le message (digitalmars.D:174223), a écrit :
> On Saturday, August 04, 2012 15:32:22 Dmitry Olshansky wrote:
>> I see it as a compile-time policy, that will fit nicely and solve both
>> issues. Just provide a templates with a few hooks, and add a Noop policy
>> that does nothing.
> 
> It's starting to look like figuring out what should and shouldn't be 
> configurable and how to handle it is going to be the largest problem in the 
> lexer...

Yes, I figured-out a policy could be used to, but since the begining of 
the thread, that makes a lot of things to configure! Jonathan would 
have trouble trying to make them all. Choices have to be made. That's 
why I proposed to use adapter range to enable to do the buffering 
instead of slicing, and to build the look up table. Done correctly, it 
can make the core of the lexer imlementation clean without loosing 
efficiency (I hope). If this policy for parsing literals if the only 
thing that remains to be configured directly in the core of the lexer 
with static if, then it's reasonable.