So... let's document dmd

[Timon Gehr via Digitalmars-d]

On 06.04.2016 06:16, Walter Bright wrote:
> On 4/5/2016 4:08 PM, Timon Gehr wrote:
>> Some additional care will need to be taken, e.g.
>>
>> /+ <backspace> <backspace>
>>
>> or even simply
>>
>> /+ comment +/
>
> that should not be a problem.
>

I think it can be for large files if the highlighter implements the 
described strategy in a literal fashion. /+ changes the lexer state at 
each position until the end of the file, and all the soon-to-be reusable 
information is replaced by irrelevant garbage by the reprocessing step. 
I think it is better to sometimes keep multiple lexer states per 
location, and maybe not necessarily run the lexer to completion each time.

Maybe this is one reason why some editors choose to close delimiters for 
you automatically by default.


One idea would be to handle all states simultaneously by keeping an 
efficient summary of the effect certain existing code snippets have on 
the lexer state. Summaries should be efficiently composable to compute 
effects (and hence the lexer state) quickly for any prefix of the code, 
while allowing code updates. I think it should be possible to fill in 
the details.