Writing a really fast lexer

[Bastiaan Veelo]

On Saturday, 12 December 2020 at 18:15:11 UTC, vnr wrote:
> On Saturday, 12 December 2020 at 16:43:43 UTC, Bastiaan Veelo 
> wrote:
>> Have you looked at Pegged [1]? It will give you the lexer and 
>> parser in one go. I'd be very interested to see how it 
>> performs on that kind of input.
>>
>> -- Bastiaan.
>>
>> [1] https://code.dlang.org/packages/pegged
>
> Yes, I know Pegged, it's a really interesting parser generator 
> engine, nevertheless, the grammar of what I would like to 
> analyse is not a PEG. But I am also curious to know the 
> performances of this tool for very large inputs.

Are you able to share the grammar? Since you plan to parse using 
recursive descent, I think there is a good chance that the 
language can be defined as a PEG. I am using it to parse Pascal, 
whose grammar was defined long before PEG was a thing.

— Bastiaan.