D grammar overhaul

[Rainer Schuetze]

Trass3r wrote:
> Am 02.04.2011, 10:11 Uhr, schrieb Rainer Schuetze <r.sagitario at gmx.de>:
>> Do we want to allow spaces and comments between these tokens?
>> - extern ( C ++ )
> Well we would have to introduce C++ as a keyword just for that purpose.
> Not worth it.

It does not have to be a keyword, but the parser can just verify there 
are no spaces, if a lexer token keeps that information. But I agree, not 
worth changing anything.

> 
>> - @ Property (safe, trusted, disable, etc)
> safe, trusted etc. aren't keywords.
> dmd handles it as ('@' identifier) and that's exactly what the grammar 
> should be to support user-defined attributes in the future.
> 

Although it is not the current behaviour, I think it would be cleaner if 
the lexer would generate a token "Property" that always starts with '@' 
and continues as an identifer. This would replace the current token "@", 
simply moving the rule "Property: '@' Identifier" from the parser into 
the lexer.

This still includes any user-defined attributes for future extensions.

I hope, we agree on !in and !is, as the current lexing introduces the 
shown parsing problems. But !is needs to be added to the IsExpression 
rules then.