[Issue 1466] Spec claims maximal munch technique always works:not for "1..3"

[Jascha Wetzel]

BCS wrote:
> Reply to Jascha,
> 
>> d-bugmail at puremagic.com wrote:
>>
>>> // thinks it's [0 ... 1], no maximal munch taking place
>>> assert (Foo[0... 1] == 0);
>>> }
>> this *is* maximal munch taking place. because of the ".." lexeme,
>> float literals are not lexemes. they are context free production rules
>> consisting of multiple lexemes. therefore "0." consists of two lexemes
>> and "..." wins the max munch over ".".
>>
> 
> But is it the the correct way to do it? (Not is is doing what the spec 
> says, but is it doing what it should be designed to do)

i think "ptr ! is null" shouldn't be allowed, because it suggests that 
"!" and "is" are separate operators and "is null" is an unary expression.
if your lexer supports lookaheads (a single character is enough), you 
can match float literals as lexemes. this is also what DMD does.