DMD as a library - recap, next steps

[RazvanN]

On Tuesday, 16 June 2020 at 12:05:11 UTC, Jacob Carlborg wrote:

> I don't think API is not the most important part, as long as 
> you can do what you need to do with the library. In my opinion 
> there's a lot of functionality that is missing. For example, 
> I've tried to use DMD as a library to do source code 
> transformation. It falls very short in this area:
>
> * AST nodes without locations

This typically happens when the compiler rewrites segments of the 
AST. It creates nodes, but doesn't bother with location since 
that code is not meant to be seen by any user.

> * Locations don't contain an end point/length
> * Locations don't contain the buffer offset
> * Indirect files are always read from disk. There's no option 
> to make a full compilation purely from memory
>
There are all valid points. I mostly thought about dmd as a lib 
as a way to analyze the AST and output relevant information (e.g. 
DCD), not as a tool to modify source code, however, I was 
expecting that the hdrgen visitor would help with that.
>
> I agree, you need a full buy in from the leadership and the 
> compiler developers. I think this will be very difficult.
>

Currently, the dmd as a library project is a state of limbo. We 
all agree that it needs to be pushed forward, but we don't know 
exactly how. This should be a good start for discussions, I guess.

>> So, how do we move forward?
>
> I think the way forward is to fork DMD to allow you to make the 
> necessary changes as you see fit without having to bother with 
> discussion and politics.
>
> If you're lucky you can merge changes from upstream easily. 
> Otherwise if you can't easily sync with upstream you can treat 
> it as a separate compiler and evolve it on its own.
>
> I've already done this, if you're interested in collaborating 
> [1].
>

I am still hoping that we can work our way with the main 
compiler, but if things don't sort out, yes, collaborating on 
your fork is definitely the best alternative.

> [1] https://github.com/jacob-carlborg/ddc
>
> --
> /Jacob Carlborg