Request for comments: std.d.lexer

[deadalnix]

On Thursday, 31 January 2013 at 12:48:03 UTC, Jacob Carlborg 
wrote:
> On 2013-01-31 13:34, FG wrote:
>
>> Do you know where you can safely cut it without having it lexed
>> beforehand? :)
>
> I was thinking that myself. It would probably be possible to 
> just cut it in the middle and then lex a few characters forward 
> and backwards until you get a valid token. Try and calculate 
> the correct index where to cut.
>
> Although I have no idea how much trouble it would given you and 
> how much you would gain.

I don't think it worth the complexity. You can lex both file in 
parallel with 2 lexer instance if you want to make things faster.