[dox] Fixing the lexical rule for BinaryInteger

[Brian Schott]

On Friday, 16 August 2013 at 22:43:13 UTC, Andre Artus wrote:
> On Friday, 16 August 2013 at 20:00:35 UTC, Brian Schott wrote:
>> I've been doing some work with the language grammar 
>> specification. You may find these resources useful:
>>
>> http://d.puremagic.com/issues/show_bug.cgi?id=10233
>> https://github.com/Hackerpilot/DGrammar/blob/master/D.g4
>
> You have done impressive work on your grammar; I just have some 
> small issues.
>
> 1. I run into a number of errors trying to generate the Java 
> code, I'm using ANTLR 4.1

I'm aware of that. If you're able to get ANTLR to actually 
produce a working parser for D I'd be happy to merge your pull 
request. I haven't been able to get any parser generators to work 
for D.

> 2. Your BinaryInteger and HexadecimalInteger only allow for one 
> of the following (reduced) cases:
>
> 0b1__ : works
> 0b_1_ : fails
> 0b__1 : fails

It's my opinion that the compiler should reject all of these 
because I think of the underscore as a separator between digits, 
but I'm constantly fighting the "spec, dmd, and idiom all 
disagree" issue.

> Same with HexadecimalInteger.
>
> 3. The imports don't allow for all cases.

https://github.com/Hackerpilot/DGrammar/issues

> 4. how are you handling the scope attribute specifier in the 
> "attribute ':'" case, e.g. "public:"?
>
> There seems to be a few more places where it diverges a bit 
> from what the compiler currently accepts.
>
> I'm not arguing for the wisdom of writing code as I am about to 
> show, but the following compiles with the current release build 
> of DMD, but may not parse with DGrammar, quite likely balk in 
> the scanner:
>
> module main;
>
> public:
> static:
> import std.stdio;
>
> int main(string[] argv)
> {
> 	auto myBin = 0b0011_1101;
>
> 	writefln("%1$x\t%1$.8b\t%1$s", myBin);
>
> 	auto myBin2 = 0b_______1;
>
> 	writefln("%1$x\t%1$.8b\t%1$s", myBin2);
>
> 	auto myBin3 = 0b____1___;
>
> 	writefln("%1$x\t%1$.8b\t%1$s", myBin3);
>
> 	auto myHex1 = 0x1__;
> 	writefln("%1$x\t%1$.8b\t%1$s", myHex1);
>
> 	auto myHex2 = 0x_1_;
> 	writefln("%1$x\t%1$.8b\t%1$s", myHex2);
>
> 	auto myHex3 = 0x__1;
> 	writefln("%1$x\t%1$.8b\t%1$s", myHex3);
>
> 	
> 	return 0;
> }

I wrote that grammar as part of my work on DCD and DScanner. My 
lexer, parser, and AST library need some more testing. Please 
download DScanner and run it with either the --ast or 
--syntaxCheck options. If you find issues, please report them on 
Github.