Fuzzed - a program to find DMDFE parser crash

[Neia Neutuladh]

On Sat, 15 Dec 2018 21:09:12 +0000, Sebastiaan Koppe wrote:
> On Saturday, 15 December 2018 at 15:37:19 UTC, Basile B. wrote:
>> I think this is what Walter calls "AST poisoning" (never understood how
>> it worked before today). And the whole parser is like this.
>>
>> This poisoning kills the interest of using a fuzzer. 99% of the crashes
>> will be in hdrgen.
> 
> As is common with fuzzing, you'll need to ensure the program crashes.
> Sometimes that requires some tweaking.
> 
> Regardless, you still have the input to investigate.

I think the point is that DMD tries to recover from parsing failures in 
order to provide additional error messages. But those parsing failures 
leave the parser in an invalid state, and invalid states are fertile ground 
for crashes.

The way to fix this is to replace the entire parser and get rid of the 
idea of AST poisoning; at the first error, you give up on parsing the 
entire file. From there, you can try recovering from specific errors with 
proper testing.