More lexer questions

[Martin Nowak]

On Sun, 12 Feb 2012 01:00:07 +0100, simendsjo <simendsjo at gmail.com> wrote:

> On 02/12/2012 12:35 AM, Timon Gehr wrote:
>> On 02/11/2012 09:59 PM, Martin Nowak wrote:
>>> Just wanted to point you to my working D lexer (needs a CTFE bugfix
>>> http://d.puremagic.com/issues/show_bug.cgi?id=6815).
>>>
>>
>> This seems to do the job:
>> constfold.c:1566
>> - if (tn->ty == Tchar || tn->ty == Twchar || tn->ty == Tdchar)
>> + if (tn->isImmutable() && (tn->ty == Tchar || tn->ty == Twchar ||
>> tn->ty == Tdchar))
>>
>> However, I don't know the compiler's internals at all, therefore it is
>> quite possible that the fix is incorrect.
>>
>>
>>> https://gist.github.com/1262321 D part
>>> https://gist.github.com/1255439 Generic part
>>
>> Bug: The lexer cannot handle /++/ and /**/ (without new line character
>> at the end).
>
> Another thing.. Using /+ and +/ in strings gives unexpected results when  
> commented out:
> /+
> auto a = "/+";

/+ comments do nest. So you have opened two levels and the comment stops  
after two pairing +/.
/* comments do not nest.

> +/
> everything from this point is commented out.
>
> /+
> auto a = "+/";
> +/ // already terminated by the string value.
>
> Is this a bug, or as designed? /++/ is meant to comment out code, so it  
> would have been nice if it was able to handle this, but I guess it would  
> complicate the lexer a great deal.