std.d.lexer requirements

[Philippe Sigaud]

On Fri, Aug 3, 2012 at 5:59 AM, Jonathan M Davis <jmdavisProg at gmx.com> wrote:
> On Friday, August 03, 2012 05:36:05 Bernard Helyer wrote:
>> If the other guys think they've got it, then I can withdraw my
>> efforts. I was just thinking I had a lexer just sitting around,
>> may as well use it, but if the other guys have it, then I'm fine
>> with withdrawing.
>
> I'm a fair ways along already. Making changes according to what Walter wants
> will slow me down some but not a lot. It's something that I've wanted to do
> for quite some time but just finally got around to starting a couple of weeks
> ago. So, I definitely want to complete it regardless of what anyone else is
> doing. It also gives me an opportunity to make sure that the spec and dmd
> match (and I've already found a few bugs with the spec and fixed them).
>
> Feel free to do whatever you want, but I'm definitely going to complete what
> I'm working on.

Look, many of us here were interested in your idea of having comments
and errors lexed as tokens.
Could it be possible to just add two static policies errorBehavior {
report, skip, ...} and comments { asToken, grouped }?
That way, a user just say;

// Walter
auto toks = Lexer!(comments.grouped)(code); // May be the default value
// Other:
auto toks = Lexer!(comments.asToken)(code);

It's just a small static if away...