A lexical change (a breaking change, but trivial to fix)

[Timon Gehr]

On 07/08/2012 12:23 AM, Mehrdad wrote:
> On Saturday, 7 July 2012 at 22:00:43 UTC, H. S. Teoh wrote:
>> On Sat, Jul 07, 2012 at 11:39:59PM +0200, Mehrdad wrote:
>>> This might sound silly, but how about if D stopped allowing 0..2
>>> as a range, and instead just said "invalid floating-point number"?
>> [...]
>>
>> I like writing 0..2 as a range. It's especially nice in array slice
>> notation, where you _want_ to have it as concise as possible.
>
> Hmm... true..
>
>> OTOH, having implemented a D lexer before (just for practice, not
>> production quality), I do see how ambiguities with floating-point
>> numbers can cause a lot of code convolutions.
>
> Yeah that's exactly what happened to me lol.
> (Mainly the problem I ran into was that I was REALLY trying to avoid
> extra lookaheads if possible, since I was sticking to the range
> interface of front/popFront, and trying not to consume more than I can
> handle... and this was the edge case that broke it.)
>

You could go like this:

switch(input.front) {
     case '0'..'9':
         bool consumedtrailingdot;
         output.put(parseNumber(input, consumedtrailingdot));
         if(!consumedtrailingdot) continue;
         if(input.front != '.') {
             output.put(Token("."));
             continue;
         }
         input.popFront();
         if(input.front != '.') {
             output.put(Token(".."));
             continue;
         }
         output.put(Token("..."));
         continue;
}

>> But I'm gonna have to say no to this one; *I* think a better solution
>> would be to prohibit things like 0. or 1. in a float literal. Either
>> follow it with a digit, or don't write the dot. This will also save us
>> a lot of pain in the UFCS department, where 4.sqrt is currently a pain
>> to lex. Once this is done, 0..2 is no longer ambiguous, and any
>> respectable DFA lexer should be able to handle it with ease.
>
> Good idea, I like it too. How about just disallowing trailing decimal
> points then?
>

+1.