std.d.lexer requirements
Christophe Travert
travert at phare.normalesup.org
Tue Aug 7 04:43:46 PDT 2012
Walter Bright , dans le message (digitalmars.D:174394), a écrit :
> On 8/7/2012 1:14 AM, Jonathan M Davis wrote:
>> But you can also configure the lexer to return an error token instead of using
>> the delegate if that's what you prefer. But Walter is right in that if you
>> have to check every token for whether it's an error, that will incur overhead.
>> So, depending on your use case, that could be unacceptable.
>
> It's not just overhead - it's just plain ugly to constantly check for error
> tokens. It's also tedious and error prone to insert those checks.
It's not necessarily ugly, because of the powerful range design. You can
branch the lexer to a range adapter that just ignore error tokens, or
throw when it meats an error token.
For example, just use:
auto tokens = data.lexer.throwOnErrorToken;
I don't think this is more ugly than:
auto tokens = data.lexer!(complex signature) { throw LexException; };
But yes, there is overhead, so I understand returning error tokens is
not satisfactory for everyone.
> I don't see any advantage to it.
Storing the error somewhere can be of use.
For example, you may want to lex a whole file into an array of tokens,
and then deal with you errors as you analyse the array of tokens.
Of course, you can alway make a delegate to store the error somewhere,
but it is easier if this somewhere is in your token pile.
What I don't see any advantage is using a delegate that can only return
or throw. A policy makes the job:
auto tokens = data.lexer!ExceptionPolicy.throwException;
That's clean too.
If you want the delegate to be of any use, then it must have
data to process. That's why I said we have to worry about the
signature of the delegate.
--
Christophe
More information about the Digitalmars-d
mailing list