Meta-programming - examples

[Andrei Alexandrescu (See Website For Email)]

janderson wrote:
> A couple of people have been asking questions about pseudo-real-life 
> examples for the new compile time code aspects of D.  I ask this 
> question for my interest as well.
> 
> Assuming that we have a good mechanism to process the input how could 
> compile time code improve your every-day proficiency?  (This is 
> specifically about the Mixin Expressions aspect).
> 
> I'll start with a few of my own.  Note my background is
> game-programming.  In my experience, we care constantly creating tools 
> to generate code because its impracticable to write and maintain it by 
> hand.
> 
> Statemachine (and AI logic) written by design and coders are common in 
> game programming.  Normally we have to fallback on a script language 
> which is never as powerful.
> 
> mixin(statemachine(
> "
>     state FireAtEnemy
>     {
>         transition EnemyRange < 10
>         {
>             goto ChaseEnemy
>         }
>         transition FacingEnemy < .2
>         {
>             goto ChaseEnemy
>         }
>     }
> 
>     state ChaseEnemy
>     {
>         ...
>     }
> 
> ");
> 
> You might imagine doing the above with if statement and arrays of 
> delegates however designers have a much harder time with that.

This example is eerily on target. A while ago I've enjoyed watching a 
talk by Shriram Khrishnamurthi on... you gotta see it:

http://technetcast.ddj.com/tnc_play_stream.html?stream_id=644

Unfortunately, it looks like DDJ does not have the audio (which was 
fantastic) anymore, so you can only see the slides (which are very good 
as well). The slides 37 to 39 knocked my socks off so hard they stuck 
onto the wall, and to this day I couldn't peel them away.

The way I'm seeing defining such DSLs is a bit more conservative than 
yours: I'd let the D compiler do most of the parsing and I'd focus on 
defining the state transitions:

class GameEngine
{
   mixin automaton!(
     State!(FireAtEnemy,
       ("EnemyRange < 10", "ChaseEnemy"),
       ("FacingEnemy < .2", "ChaseEnemy")),
     State!(ChaseEnemy,
       ...)
   )
}

It's amazing to think that D will allow this (actually already allows 
with a slightly clunkier syntax). I'll make sure I'll be barefoot that 
day. :o)

> 3D Object rendering
> 
> //Render a 3D model.  For instance this may generate the vertexbuffers 
> and textures directly into code, or may extract some of the data from 
> the 3Dmodel.obj and load the rest at run-time.
> mixin(renderModel(import("3Dmodel.obj")));
> 
> Transforming Textures
> DXDtexture* texture = mixin(textureToD3D(import("3Dmodel.bmp")));

Sounds great, but no expertise to comment on that.

> Simplify saving/loading of arbitrary structures.
> 
> //Only write this once.  Things are re-ordered for best fit in memory, 
> localization ect...
> mixin(Serailzable("Ship",
> "
>     struct Part
>     {
>         float firePower;
>         char[] Model;
>     }
> 
>     struct Ship
>     {
>         bool Friendly;
>         Part Parts[];
>     }
> ");
> 
> ... Later
> 
> Load("Ship", "Ship.asset");

Yah, this has been pointed out indeed. Also add remote procedure calls. 
Generating all of the marshaling/unmarshaling code (for various 
standards!) will be trivially done by libraries. I actually see PyD as a 
great first application of that concept that will spark many others.

> GUI -> not sure about this one as I can't come up with a format thats 
> much neater then straight code. I'll post it anyways.
> 
> mixin(MyGUI(
> "
>     Button "OK", 10, 30
>     {
>         Exit;
>     }
> ");

Great. The problem with GUI programming has always been that programming 
language is not a natural vehicle for specifying forms and stuff. Many 
frameworks have relied on some framework for code generation and 
impedance matching (Tcl/Tk, Windows resources, ...) with various levels 
of awkwardness. Due to its compile-time manipulation abilities, D will 
make it trivial to bind GUI resources to D code.

The way I see this happening is more along the lines:

mixin(import "inputform.rc")

The file can be generated and edited with other tools (e.g. gui 
designers). That's why I insisted with Walter that strings should be 
importable from files. This effectively makes you benefit of the dual 
cake paradox, in that at the same time you have the advantage of 
separate files for separate languages, and the advantage of seamless 
integration between them.

> Multi-threading
> 
> The suggestion with multi-threading I came up with a little while ago:
> 
> int[] array ...
> int result = 0;
> mixin(threadIt(
> "
>     foreach(A a; array)
>     {
>         result += a;
>     }
>     combiner //If necessary?
>     {
>         result = result[0] + result[1];
>     }
> ");
> 
> Compile time checking of code for coding standards or what have u:

I'm not understanding this.

> mixin(CheckCode("
> 
> ect...
> 
> 
> ");
> //Note that there should be a way to disable CheckCode in the general 
> case to save compile time

I guess this is an obvious:

mixin(CheckCode(import "program.d"));

A file containing a mixin for each program in a project can be generated 
by the build process.

> There's heaps of other stuff, but that would make this post way to long.

There's always the opportunity of a next post :o).

> Personally I see this new mixin format as giving us the power to extend 
> D in whatever way meets our goals.  As Walter adds more features some of 
> them will be rarely used (law of demising returns) because they are 
> special case.  Mixin Expressions circumvent this because now the 
> programmer has the power to create the language features they need.

Yah indeed. This has been a source of disagreement in the past between 
Walter and myself. He used to favor compiler-wired features, while I 
envisioned an infinitely configurable language with an exceedingly small 
core. Time has past, and we both learned some of the advantages of the 
other's approach.

> The only downside I see is compile-time, which I hope can be solved with 
> a smart caching system.

Stay tuned. Compilation times will come down dramatically. Walter is up 
to something pretty cool, but it will take a while. Advice: don't wear 
socks when reading d.announce :o).


Andrei