dmd Lexer and Parser in D

Timon Gehr timon.gehr at
Sat Feb 4 10:24:56 PST 2012

On 02/04/2012 06:39 PM, Zach Tollen wrote:
> On 2/4/12 6:59 AM, F i L wrote:
>> Very cool. I was talking with someone on the IRC about the
>> possibility/difficulties of making DMD's parser/lexer/AST stay open in
>> memory with protocols designed for IDE code-completion communication. It
>> would be ideal to have an IDE's intellisense automatically update with
>> DMD semantically.
> This is my thinking too. One good thing about having cut the program is
> that it's a much lighter weight now, and I did it because I thought,
> well, maybe once it's paired down, I can actually steer it toward IDE
> functionality. For example, you could really cut out a lot of the
> members of the data structures which only point to backend functionality
> anyway.
> Even if the whole project fails I won't regret doing it because I
> learned a lot about D in the process.
> What I'm really wondering is if you wanted a program which helped you
> edit the syntax tree directly and only produced a text file for saving
> and running, what kind of data structure would you like to have
> representing the syntax tree? Without knowing anything else, I guessed
> that it would be nice to have something resembling the official D
> parse-tree.
>> Unfortunately the conclusion was that it would be to difficult an
>> undertaking to be realistic, since DMD is designed to be run-and-done
>> (also something about "Walter code" :-)).
> I was wondering if you couldn't take a parse-tree data structure and
> deparse (disparse?) it back to formatted program code so that you could
> see what you were editing? As unrealistic as that sounds, I'm
> sufficiently attracted to the idea that I'm investigating it with an
> open mind.
>  > But maybe a rewrite/port of DMD, especially one written in D, might
>  > be able to be reworked with this
>  > goal in mind? How complete is DDMD?
> This is exactly what I'm aiming at. My basic hopes for its being
> possible are the comforting notion that the huge part of dmd is actually
> the stuff I threw out! The goal would be to construct the front end (of
> a front end) which was at least theoretically capable both of allowing
> code editing, and of translation to a more backend-friendly data
> structure. If that's not possible, then I'm stuck with this thought that
> you edit the tree, then the IDE reverses the parse back into ordinary
> code for saving and compiling.
> If anybody can refer me to any examples and demonstrations of this type
> of code-editing, please do. As someone new to programming I'm really
> wondering why, if the program itself is understood by the computer as a
> tree, why do I have to edit a text file instead of a tree?
> Zach

You are: The source file can be seen as the representation of a tree 
structure, and if you read the source you group the characters in a 
tree-like way in order to understand what it is saying. Anyway, this is 
true for any language. Your post could be parsed into a tree structure too.

You might want to have a look at lisp. Its syntax is a straightforward 
description of the parse tree.

More information about the Digitalmars-d mailing list