Metacode mechanics

[Don Clugston]

Georg Wrede wrote:
> Back to the issue at hand:
> 
> What I'm still saying, is that (even though Walter and Don have made 
> completely kick-ass progress here), there's a glass ceiling waiting for 
> us at this rate. By the time we have evolved the D meta business into 
> something that has cleared all major obstacles, it will be a hard to 
> use, cumbersome, and unobvious metalanguage. (( but still a hell of a 
> lot nicer and of course much more powerful than C++ templates ever ))

I'm not sure about this. What do you see as the major obstacles?
I had a long list of them once, it's very short now.
Note that almost all of the improvements which have been made, have made
metaprogramming *more* similar to ordinary D, not less.

The main reason D metaprogramming is so much simpler than C++, is that 
we have "static if" instead of using partial template specialisation.

I'll just summarise the changes I proposed that Walter has implemented:

* strings and reals as template value arguments.
  ---> makes metafunctions able to have a greater variety of parameters,
       and therefore more like normal D.
* Improved compile-time constant folding, most notably [] and [..]
  ---> makes compile-time arrays more similar to runtime arrays.
* template quirks fixed. Most notably scope of static if.
  ---> less weird workarounds
* .mangleof property.
  OK, this one's a bit weird. But, it does establish a closer link 
between the two languages: it means that any type can be expressed in 
the run-time language.

The remaining changes that are accepted for D 1.0
* implicit function template instantiation
--> makes invoking a template function look the same as invoking a 
normal function.

and beyond D 1.0
* array literals
  ---> makes compile-time arrays more similar to runtime arrays.

The only other metaprogramming functionality I'd really like, but don't 
have, is full compile-time reflection, and possibly the ability to 
convert strings into variables.

I've spent a few months pushing the boundaries of what's achievable with 
D, and frankly, I'm running out of ideas. Practically everything I've 
wished I could do in C++ seems to already be possible in D, with the 
exceptions listed above.

In C++, metaprogramming feels like programming in an extremely 
impoverished functional programming language. But, in D, it feels like 
programming in a subset of D -- (you can only assign to variables at 
construction, you must use recursion for looping) -- a metafunction can 
be trivially converted to a runtime function (change 'static if' to 
'if', remove all of the "const"s from the variable declarations, remove 
all the "!", and change  "template abc(char [] str)" to abc(char [] 
str)"). Normally you don't use recursion much in a D runtime program, 
but it will work.

So, I don't see any signs of D metaprogramming becoming more cumbersome 
than it is now. I really don't see grounds for pessimism.
BUT...
it's probable we could do better. I feel, though, that we should be 
aiming for a metalanguage that is more similar to ordinary D, rather 
than try to completely separate them.

> Thus, it would behoove us to start examining the features and properties 
> of a from-ground-up designed meta level language for D. (Definitely not 
> ever even intended to anything earlier than D2.0!)
> 
> A year ago this was a pipe dream. But now, with the experience we gain 
> from the existing meta language, and (foremost) since I'm not alone 
> anymore in thinking that such an undertaking is actually within reach, 
> we really could start outlining it now!
> 
> ---
> 
> So I suggest that we:
> 
>  - continue developing the current D metalanguage
>  - continue to have both Don's and Phobos' regexps
>  - in this NG start the development of requirements for KBDM
> 
> KBDM being Kick-Butt D Metalanguage. :-)

I'll be interested to hear any further ideas you have.