[GSOC Draft Proposal] ANTLR and Java based D parser for IDE usage

[Luca Boasso]

Sorry for my late draft proposal, I'm currently moving so I didn't
have enough time this days.
I would be glad to have your opinion.

Thank you

<DRAFT PROPOSAL>

Rationale
---------

There are different IDEs for the D programming language. The purpose of this
project proposal is to write a parser for the D programming language (v1 and v2)
that is tailored for IDEs needs. The new parser will be designed to be modular
and abstracted from any particular IDE implementation detail, so that it can be
used in different IDEs or with tools that need an abstract syntax tree of the
source code for further analysis.
Particular care will be taken to integrate the new parser with the DDT
Eclipse-based IDE so that this project will be useful in the short-term.
The DDT project needs a new parser up-to-date with the latest D syntax, with
a better error recovery and improved performance.
Thanks to this integration it will be possible to understand the appropriate
interface for the parser so that in the long-term the same code could be used in
different projects.

I will use the ANTLR parser generator for this project. This parser generator
has been proven to be a valuable tool with advanced features for tree
construction and  tree manipulations that cuts development time [1]. The LL(*)
parsing algorithm  on which ANTLR is based upon allows efficient parsing of
complex grammar with good error handling and unrestricted grammar actions [2].

The use of a parser generator allows the creations of parsers in different
programming languages. This project will focus on the creation of a Java parser.
Since ANTLR support many target languages [3], it will not be so difficult to
generate a parser in the original implementation language of the IDE.
Eg. Generate a C++ parser for the D language that will be used in the IDE
written in C++.

Furthermore, updates of the D grammar are reflected in a more convenient way
through modifications of the ANTLR grammar of D, than through a modification of
a hand-written parser.
In particular, one of the problems faced by DDT developers was to keep their
parser up-to-date with the reference one (DMD parser) [4].
It is time-consuming and error-prone to manually port the DMD parser written in
C++ to another language, instead most of the modification will be handled by
ANTLR.

In addition, easy modification of the D language syntax encourages
experimentation for the benefit of the language's evolution.


Finally in the process of writing a new parser eventual misunderstanding or
inconsistency of the D language reference and documentations will be addressed.
A good set of test will be created to guarantee the compatibility of the new
parser with the official language definition and the DMD parser.

Timeline
--------
This is a tentative timeline to be further discussed with the help of the
community. I'm committed to dedicate substantially to this project
knowing that I also have to pass some exams. I estimate that I could spend
initially approximately 25h/week. After the exam session I will work full-time
on this project.

* April 25 – May 23: Community Bonding Period
  Since I am new in the D community I will spend some time learning how to
  contribute while following the guideline of the community and the DDT project.
  I will start reviewing the language reference asking for clarifications when
  needed. Once I have got an overall understanding I will write the production
  rules of the D grammar (v1 and v2) in the ANTLR grammar notation (similar to
  EBNF).

* May 23 – July 11: Developing phase I
  The correctness of the parser is of paramount importance. I will create
  many tests to exercise the parser (at this point just a
"recognizer") obtained
  as output from ANTLR.
  Once I am confident with the parser conforms to the language reference and
  recognizes the same language as the parser in DMD, I will enhance it with
  AST construction rules.
  At this point, I need to discuss with the DDT team the type of AST that has to
  be built for IDEs purposes, and confirm which annotations are most useful
  (eg. source ranges).

* July 11 – August 15: Developing phase II
  In this phase I will create unit tests to verify the correctness of the
  generated trees and I will focus on the integration of the parser with the DDT
  project.
  In the remaining time I will provide good error recovery to the parser and I
  will improve the overall performance.

* August 15 - August 22: Final phase
  I will use this last week to polish the code and improve the documentation.
  As a final task, I will think about how support for incremental parsing can be
  added in the future.

About me
---------

I am Luca Boasso, I am a computer engineering student of Polytechnic University
of Turin [5]. I won a scholarship for a double-degree program and obtained the
first master in France under Telecom ParisTech [6] in 2010.
After having an internship in Panasonic R&D in Cupertino [7], I am currently
finishing my last semester in Italy to receive my second master.
I have been always had a passion for programming language design and
implementations. I study and use in my spare time several programming languages.
I have become a recent supporter of the D programming language and I am amazed
by its expressiveness, power and net improvement over C++.
In my master thesis, I designed a domain specific language for SAP lab France
and I wrote several parsers by hand and also with the help of ANTLR.

Contact Information
-------------------
Skype: luca_378
Email: luke.boasso at gmail.com