Is metaprogramming useful?

[Jarrett Billingsley]

"Steve Horne" <stephenwantshornenospam100 at aol.com> wrote in message 
news:tqrmm25d9217h5r45hem4jnp2g1hi6oh7o at 4ax.com...

> When doing the metaprogramming, the compiler is basically acting as an
> interpreter as opposed to a compiler. It is nothing more advanced or
> freaky than that.
>
> A lot of work is already done in interpreted languages already. In
> fact, since that term is often applied to languages that actually
> compile to virtual machine bytecode (e.g. Python) and since other
> languages that compile to virtual machine bytecode are considered
> compilers (e.g. Java) there is no clear line between compilers and
> interpreters these days. Interpreters often do a bit of pre-compiling
> (to intermediate code), and compilers often do a bit of interpreting
> (precalculating constant expressions, as well as metaprogramming).
>
> The reason C++ metaprogramming is slow is (1) because it has never
> been a big priority for compiler developers, and (2) because C++
> template metaprogramming forces programmers to work in indirect and
> inefficient ways, substituting specialisation for simple conditionals
> and recursion for simple loops.
>
> Support metaprogramming properly and it need be no slower than, for
> instance, using a scripting language for code generation - an
> annoyingly common thing. In fact, the compiler could simply generate a
> compiled code generator, run it, and then pick up the output for the
> next compilation stage if performance was that big a deal. This could
> give better performance than writing code generation tools in C++,
> since the compiler knows its own internal representations and could
> create compiled code generators that generate this directly rather
> than generating source code.

A little off the topic, but not really, as it spins off of the idea of a 
compiler as an interpreter.  I always thought it would be an interesting 
exercise to make a language where metaprogramming is not only possible, but 
nearly as full-featured as the actual code.  That is, templates (or whatever 
they'd be called, because they'd be far more advanced) would be a script for 
the compiler, which could even be compiled to bytecode.  The input would be 
language constructs -- symbols, statements, expressions -- and the output 
would be code which could then be compiled.  It'd basically be a scriptable 
compiler.

The problem that I always run into in this thought experiment is the syntax. 
I mean, for a lot of cases, template syntax is great.  When you're writing a 
simple type-agnostic stack class, you just need to be able to substitute T 
for the type of the data everywhere; it's simple, straightforward, and easy 
to understand.  Using another syntax for that would be kind of clumsy.

Maybe, then, this metaprogramming language would have several different 
constructs, just as the real language does.  It might have a templating 
system for replacing types, symbols, and constants in code.  It might then 
have other constructs for dynamic code modification and generation.

// The namespace is where the function will be generated
metafunction WrapFunction(symbol func, namespace owner = func.namespace)
{
    assert(func.isFunction, "WrapFunction: '" ~ func.nameof ~ "' is not a 
function");

    // Generation of identifiers from strings
    symbol wrappedName = symbol(func.nameof ~ "_wrapped");

    // Create a new function, and set its first param name to 's'
    int function(MDState) wrappedFunc = new function int(MDState);
    wrappedFunc.params[0].name = identifier("s");

    // An array of symbols for calling the real function
    symbol[] params;

    // The code for the generated function
    Statement[] code;

    foreach(i, param; func.params)
    {
        // Add the new param name to the symbol array
        params ~= symbol("param" ~ i);

        // VarDeclaration takes type, symbol of the name to declare,
        // and an initialization expression.
        // DotExp takes the two sides of the dot.
        // TemplateFunctionCallExp takes a name, a list of types, and a
        // list of params (not used here).
        code ~= new VarDeclaration(param.typeof, params[$ - 1],
                new DotExp(symbol("s"),
                new TemplateFunctionCallExp(symbol("pop"), param.typeof)));
    }

    // Get the function return (if any)
    if(is(func.returnType == void) == false)
    {
        // Get the return type in the local "ret" and push it, then
        // return 1
        code ~= new VarDeclaration(func.returnType, symbol("ret"),
                new CallExp(func, expand(params));

        code ~= new CallExp(new DotExp(symbol("s"), symbol("push")), 
symbol("ret"));
        code ~= new ReturnExp(1);
    }
    else
    {
        // Just call the function and return 0
        code ~= new CallExp(func, expand(params));
        code ~= new ReturnExp(0);
    }

    // Set the wrapped function's code to the code array
    wrappedFunc.code = code;

    // Put the function in the owner namespace's symbol table
    owner[wrappedName] = wrappedFunc;
}

...

int fork(int x, float y)
{
    writefln("fork: x = ", x, " y = ", y);
    return 65;
}

WrapFunction!(fork);

That call would produce the code:

int fork_wrapped(MDState s)
{
    int param0 = s.pop!(int)();
    float param1 = s.pop!(float)();
    int ret = fork(param0, param1);
    s.push(ret);
    return 1;
}

;)