Is metaprogramming useful?

[Kirk McDonald]

Frank Benoit (keinfarbton) wrote:
> In generic programming we can do wonderful things like container classes
> and template functions.
> 
> Using more advanced feature will result in metaprogramming, like the
> compile time regex.
> 
> I wonder if compile time regex is really a thing of practical use.
> 
> If I a lot of compile time regex pattern, it is very nice to have
> compiler error for invalid patterns.
> 
> But disadvantages are:
> - compile time is increasing fast
> - code size is increasing with every new pattern (is it?).
> 
> With a parser like Spirit in C++ it would be the same. In the Spirit
> FAQ, a 78 rule parser was reported with 2 hours compile time.
> 
> Well, D might be faster, but it shows that the compile time can increase
> very fast.
> 
> In both cases an external regex or parser generator would make more sense.
> 
> Now my questions:
> 1.) Is metaprogramming really useful or only kind of hype?

Naturally, I will speak to what I know, which is Pyd. A goal of Pyd is 
to wrap as much of the D language as possible, as easily as possible. 
This requires meta-programming techniques: Template functions, function 
meta-info, typeof, tuples.

Thanks to these techniques, I can completely wrap a D function without 
knowing its return type, argument types, or even its name, by just saying:

def!(foo);

"Wrapping" a function implies generating a function (with C linkage) 
that accepts (in the case of the Python API) some PyObject* arguments 
and returns a PyObject*; somehow calls the function with the PyObject* 
arguments; and somehow converts the function's return value to a 
PyObject*. All of this can occur automatically, given only an alias to 
the function. The generated code is probably only slightly slower than 
if the wrapper had been written by hand (though I have yet to test Pyd's 
performance). It might even be faster, if the hand-written code uses 
PyArg_ParseTuple to convert the arguments. (Which determines the types 
to convert to at runtime, using a format string. Pyd determines the 
types at compile-time.)

And, regarding compilation times: Pyd compiles nigh-instantly. 
Boost::Python is notorious for long (sometimes in the range of hours) 
compilation times. D seems to be just plain better at this than C++.

> 2.) Are there examples where a metaprogramming solution is the best way
> to solve it?

I would say that using Pyd (or even Boost::Python) is much more pleasant 
than the raw Python/C API.

As soon as you start working with metaprogramming, you start wanting 
compile-time equivalents of things you have at runtime. I've started 
working on a compile-time writef-style formatter, for instance. (Due to 
bug 586, it only works with types at the moment, and not other 
compile-time values like ints and function aliases. But you can specify 
field widths, and so forth.) This thing is based heavily on stuff in Don 
Clugston's meta library (have you had the same idea, Don?), and serves 
to unify many of the string-conversion templates that are floating 
around (Don's Nameof, that itoa template, and so on).

-- 
Kirk McDonald
Pyd: Wrapping Python with D
http://pyd.dsource.org