DMD as a library - recap, next steps
WebFreak001
d.forum at webfreak.org
Tue Jun 16 10:48:13 UTC 2020
On Tuesday, 16 June 2020 at 04:13:10 UTC, RazvanN wrote:
> A few years ago, I worked on a project to refactor the dmd
> codebase so that it becomes easier to be used as a library.
> This has the advantage that tools that use dmd-as-a-lib will
> rely on the latest working compiler. From that perspective, I
> made several PRs:
>
> [...]
>
> Me and Edi are in the position where we can use bachelor
> students to do the heavy-lifting on helping this project to
> cross the finish line, however, it is critical that the Dlang
> Foundation leadership has a clear direction/vision on this.
>
> So, how do we move forward?
>
> Cheers,
> RazvanN
very cool, thanks for all the work! I think however increasing
the compilation time significantly is a major no-go, at least
when it affects all code, expect of course if it allows us to do
faster things like incremental compilation.
For architecture ideas you might want to check out how Microsoft
implemented their Roslyn compiler platform:
https://github.com/dotnet/roslyn/wiki/Roslyn-Overview
Just AST and visitors is not the most useful on its own, we have
libdparse for this already which works fine too. Much more
interesting is the semantic analysis.
If there is one thing I would want exposed by dmd for anything,
being completion, dynamic linting, navigation, etc., I would
really really want a symbol API. Much like dsymbol one
incremental database of all defined symbols (modules, types,
aliases, parameters, template parameters, variables, etc.) with
references, definitions, types (of variables and parameters),
names and all traits information. This database would contain all
symbols in the entire compilation unit, be aware of scopes at any
given point and be able to incrementally update by
adding/removing or changing files.
The semantic analysis needs to be incremental here too though, so
symbols would need some kind of dependency graph for things using
mixin or templates. Also it would be difficult to extract
information from scopes like `version (Foo)` where version is not
Foo.
But just this symbols database and some APIs to query it per
location, per file or per symbol name would be enough to
implement nearly all features a user would expect from tooling
and more.
Otherwise raw token access and AST visitors is all you really
need to implement the rest like formatting, highlighting, static
linting and refactorings. It's important that you can somehow
recover the whitespaces and comments from tokens for refactoring
and formatting though!
For other use-cases, like a REPL, exposing APIs to the executable
generator would also be cool.
So if we have a symbols API I'm happy and I think that will be
the goal of any DCD replacement program too :p
Keep up the good work on this!
More information about the Digitalmars-d
mailing list