What are AST Macros?

Thu Jul 8 18:42:58 PDT 2010

"Jonathan M Davis" <jmdavisprog at gmail.com> wrote in message 
news:mailman.301.1278636124.24349.digitalmars-d at puremagic.com...
> Periodically, the term AST macros come up in the discussions here as a 
> possible
> addition to the language - they even get discussed briefly in TDPL as a 
> possible
> replacement for template mxins. However, I don't think that I've ever seen 
> an
> actual explanation for what exactly is meant by the term AST macro. This 
> raises
> two questions
>
> 1. What are AST macros and how do they differ from C-style macros? We 
> obviously
> aren't going to be adding macros like that to D, since that would be 
> dangerous.
> But what would a macro be then if it's not a textual replacement? The best 
> I can
> think of would be that you'd have to indicate the macros to be replaced by
> surrounding them with macro() or something rather than letting any and all 
> text
> that matches  be replaced. So, I really don't know what AST macros are 
> supposed
> to be other than they're macros of some kind and better than C-style 
> macros.
>
> 2. What does AST stand for? The best that I can come up with for what it 
> could
> stand for would be Abstract Syntax Tree, which is a nice, Computer 
> Sciency,
> compiler-related term, but I haven't a clue how that would relate to 
> macros. So,
> maybe an answer to the first question would answer this one as well, but 
> an
> explanation would be nice.
>
> - Jonathan M Davis

Short answer: Reflection/introspection on steroids.

Slightly longer answer:

You're right, AST is "Abstract Syntax Tree". As for what that actually 
means: You know the HTML DOM? If you have the DOM for a specific HTML page, 
that DOM is a (mutable) representation of the HTML page's AST. If you 
manipulate the DOM (ie, DHTML), then that's basically AST macros.

Except for one thing: HTML is merely markup and D is fully turing-complete. 
So imagine that the browser *also* has a DOM for the JavaScript itself, so 
the JavaScript can manipulate itself through a DOM...that's what we're 
talking about (although it'd be compile-time in our case). So yea...clearly 
powerful stuff.

Purely hypothetical example:

Suppose you have this code:

module foo;
void main(string[] args)
{
    int i;
    i = 7 + 4 * 3;
}

The AST might look something roughly like this (I'm representing it here in 
XML, so I don't have to try to draw an ASCII-art graph):

<module name="foo">
    <function name="main" type="void">
        <params>
            <param name="args" type="string[]" />
        </params>
        <body>
            <declaration name="i" type="int" />
            <assignment>
                <lvalue>
                    <variable>i</variable>
                </lvalue>
                <rvalue>
                    <add>
                        <literal>7</literal>
                        <multiply>
                            <literal>4</literal>
                            <literal>3</literal>
                        </multiply>
                    </add>
                </rvalue>
            </assignment>
        </body>
    </function>
</module>

(Or it might look totally different then that, depending on how the AST 
system is defined, but you get the idea.)

Now imagine an API that exposes a mutable version of that tree (or something 
similar to it) at compile-time.

Also, if you know what a parse tree is, an AST is similar to that, but 
generally a bit more human-understandable and semantics-oriented, whereas a 
parse tree is purely a literal representation of the formal syntax - no 
semantics all, and rather awkward to deal with.

Another interesting thing: I've never really used LISP, but my understanding 
is that this sort of thing basically forms the heart of LISP and is the 
reason LISP is considered so powerful. (AIUI, LISP was invented purely as a 
hypothetical example of this exact sort of thing, and then somebody went and 
actually implemented a real interpreter for it.)