Adela Vais - SAOC Milestone 4 Update 3 - Dlang GLR Parser for GNU Bison

[Adela Vais]

On Thursday, 7 January 2021 at 23:30:29 UTC, Adela Vais wrote:
>>>>>>>[...]

Hello!

As of the last update:

- I modified the calc example to use lookahead correction. Using 
my last week's work on an unmodified calc example, I modified 
this program to use a push-parser.

- I made some small style fixes in the examples and submitted 
them for review along with the std.conv.parse modification 
started last week. The enhancement to parse was introduced in 
Dlang v2.095.0, so we will support both versions for the calc 
example: the one demonstrating the new parse feature, and the one 
using the old code. [1]

- I started working on token constructors. [2] This feature works 
in the C++ parser only using the '%define api.value.type variant' 
option.

By default, the user must write a union containing all the 
different types of values needed for yylex(), and then use them 
when declaring the tokens:

%union {
   int ival;
}
[...]
%token <ival> NUM "number"

Variant allows the user to simply write:

%token <int> NUM "number"

D does not support this feature, but I plan to introduce it in 
the near future (not necessarily during SAOC).

 From yylex(), the user must return a Symbol by calling its 
constructor. This constructor does not check if the TokenKind and 
the value correspond in any way, so the user can return 
Symbol(TokenKind.NUMBER, "I am a string") and the error will be 
caught much later in the program, and, of course, at runtime.

As a solution to this, the C++ parser provides the option of 
calling the make_<token_name> function (example: make_NUMBER), 
which calls the Symbol constructor with the correct arguments, 
and generates compile-time errors. I want to provide this feature 
for D (modified to be called as Symbol.NUMBER(someNumber)).

The C++ parser generates the make_ functions with M4, so all the 
functions are put in a header file for the user. I want to limit 
the space occupied by these methods by generating them using D. 
[3] I created a version that does not use variant, as a proof of 
concept that this task can be done from D. But once I add support 
for variant, the token constructors should work without 
modifications.

The plan for the next week of #SAOC2020:
- Continue working on the above. Push-parser next steps:
     * I have to do more correctness and speed tests for both 
versions (with and without lookahead correction).
     * Start integrating it in the D backend.

[1]: https://github.com/adelavais/bison/tree/calc-example-fix
[2]: https://github.com/adelavais/bison/tree/token-constructors
[3]: 
https://github.com/adelavais/bison/blob/token-constructors/tests/testsuite.dir/582/calc.d#L544