Writing a JFlex lexer for D - have an issue with cycles

[FatalCatharsis via Digitalmars-d]

On Monday, 23 January 2017 at 01:40:28 UTC, Chris Wright wrote:
> On Mon, 23 Jan 2017 00:46:30 +0000, Profile Anaysis wrote:
>> But this is ok because parsers are recursive in nature, so you 
>> just have to have your rule be able to terminate in a logical 
>> way.
>
> Except it's part of the lexer, and most people (such as those 
> creating the JFlex lexer generator) assume that no lexer needs 
> to be recursive.

I was under the impression that a lexer could not be recursive by 
nature, that it was up to the parser to handle recursion and 
nesting productions.

Can you provide articles/examples showing how a lexer can handle 
recursive cases?