DIP 50 - AST macros

[Jacob Carlborg]

On 2013-11-12 21:03, Joseph Cassman wrote:

> After using string mixins for a while I have come to feel much like your
> statement in (1). I like the power and flexibility of string mixins.
> Text and strings are understandable and straightforward to manipulate.
> This is a clear win in my opinion.
>
> There are also some things I find difficult at the moment.
>
>   - One is the process of what to do when things do not go as expected.
> To debug I currently use pragma statements and try to visualize what is
> going on in my mind. This is do-able. But I feel it could be better.

If we got macros we should really have a good way to debug them. At 
least a flag to shows how the macros are lowered.

>   - It would be nice to be able to use some more D language concepts to
> construct the string mixin inside the template. For example, sometimes I
> would like to use a loop to construct the result.
>
> If these two items could be addressed I would be pretty satisfied with
> using string mixins as they are.
>
> One win that the AST macro concept provides that string mixins does not
> is manipulation of code syntax trees (either at compile or run time). It
> would be nice to combine the various code manipulation ideas that have
> been tossed around before into a single module, say std.meta or
> std.reflection. In addition to combining the various current code
> manipulation functionality into a single api (e.g. __traits,
> std.traits), it could provide functionality to work with code-in-use.
> For example, it would be pretty cool to be able to do the following (the
> actual syntax for constructing the code is not that important to me, but
> the underlying functionality for manipulating code is).
>
> sample.d
>
>      import std.meta;
>      void main(string[] args) {
>          auto e = construct("a + %s", args[1]);
>          writefln("%s", e(5));
>          auto f = construct(e + "b");
>          writefln("%s", f(?, 10).toDebugString);
>          writefln("%s", f(5, 10));
>      }

Yes, but the problem with string mixins is that you need to reparse a 
lot of code. With string mixins it becomes:

1. The compiler parses the code and creates an AST (fine so far)
2. Does type checking an other stuff
3. Library code parses a string into an AST
4. Library code manipulates the AST
5. Library code converts the AST back to a string
6. The string is interested with a string mixin at the calling context
7. The compiler parses the code and creates an AST
8. Does type checking an other stuff

With AST macaros you can avoid many of these steps:

1. The compiler parses the code and creates an AST
2. Does type checking an other stuff
3. Library code (macro) manipulates the AST
4. Compiler inserts the manipulate AST into the calling context
5. Does type checking an other stuff

Instead of eight steps we get five steps. We remove the unnecessary 
conversions between AST's and strings.

-- 
/Jacob Carlborg