Writing a JFlex lexer for D - have an issue with cycles

[FatalCatharsis via Digitalmars-d]

On Monday, 23 January 2017 at 00:46:30 UTC, Profile Anaysis wrote:
> The real issue is ambiguity. Any time you have a cycle you must 
> be able to get out of it and so your rules must be organized so 
> that one always checks to see if termination has occurred 
> before checking for nesting. If you allow, say } as an element 
> of a stringliteral then it will be ambiguous as the grammar 
> will try to match it, when it is meant to be a literal.

This brings up another question for me. Isn't the token string 
production ambiguous by itself?

q{ tokens }

How does it know whether the input "}" is another token or the 
terminator on a token string?