DIP 50 - AST macros

[Joseph Cassman]

On Tuesday, 12 November 2013 at 11:06:17 UTC, Don wrote:
> On Tuesday, 12 November 2013 at 09:55:20 UTC, Walter Bright 
> wrote:
>> I forgot to mention that "expression templates" can be used in 
>> D in an equivalent manner that they are used in C++,
>
> They are crippled compared to C++, because you have no control 
> over the return type of opCmp and opEquals, which means that 
> you can't have expression templates involving comparisons. 
> That's extremely limiting.
>
>
> My feeling with AST macros is:
>
> (1) AST macros are basically syntax sugar for string mixins. 
> The one and only thing that string mixins have in their favour, 
> is that they have almost no syntax. But that is actually a 
> *huge* plus.
>
> Because it's syntax sugar, an AST macro syntax would be most 
> convincing if you took some existing string mixins from various 
> real projects, and show how beautiful they would become with 
> macros.
>
>
> (2) The big functionality we don't have, is comprehensive 
> reflection. I'd like to see more thought in that area. It seems 
> challenging to define reflection in a way that doesn't expose 
> compiler internals.
>
> The question I find particularly interesting is, "if we had 
> macros, would reflection look different?"

After using string mixins for a while I have come to feel much 
like your statement in (1). I like the power and flexibility of 
string mixins. Text and strings are understandable and 
straightforward to manipulate. This is a clear win in my opinion.

There are also some things I find difficult at the moment.

  - One is the process of what to do when things do not go as 
expected. To debug I currently use pragma statements and try to 
visualize what is going on in my mind. This is do-able. But I 
feel it could be better.

  - It would be nice to be able to use some more D language 
concepts to construct the string mixin inside the template. For 
example, sometimes I would like to use a loop to construct the 
result.

If these two items could be addressed I would be pretty satisfied 
with using string mixins as they are.

One win that the AST macro concept provides that string mixins 
does not is manipulation of code syntax trees (either at compile 
or run time). It would be nice to combine the various code 
manipulation ideas that have been tossed around before into a 
single module, say std.meta or std.reflection. In addition to 
combining the various current code manipulation functionality 
into a single api (e.g. __traits, std.traits), it could provide 
functionality to work with code-in-use. For example, it would be 
pretty cool to be able to do the following (the actual syntax for 
constructing the code is not that important to me, but the 
underlying functionality for manipulating code is).

sample.d

     import std.meta;
     void main(string[] args) {
         auto e = construct("a + %s", args[1]);
         writefln("%s", e(5));
         auto f = construct(e + "b");
         writefln("%s", f(?, 10).toDebugString);
         writefln("%s", f(5, 10));
     }

-----

> ./sample 2
> 7
> Numerical Expression: a + 2 + 10
> 17

Implementing the "expression tree" side of AST macros seems like 
it could be done as a library, which would be a win for 
maintainability and cross-compiler portability. There is probably 
required, compiler-related functionality that I cannot see making 
this unfeasable or possibly difficult to do. Still, looks good to 
me on paper at least. Perhaps going this route could gain the 
good functionality of AST macros while also making use of the 
already implemented features?

Seems to me like some of these ideas tie into previous 
discussions. Here are two I remember. I am sure there are others.

http://forum.dlang.org/thread/juf7sk$16rl$1@digitalmars.com
http://forum.dlang.org/thread/mailman.1716.1340388570.24740.digitalmars-d@puremagic.com

Joseph