CTFE vs. traditional metaprogramming

[Jarrett Billingsley]

On Fri, Oct 9, 2009 at 3:49 PM, Andrei Alexandrescu
<SeeWebsiteForEmail at erdani.org> wrote:
> Thanks!
>
>
> I plan to add more text at the end of the chapter that discusses the
> opportunities of CTFE. Walter revealed to me that CTFE, particularly now
> after it's been improved by leaps and bounds by Don and by Walter himself,
> could obviate a lot of the traditional metaprogramming techniques developed
> for C++.
>
> One question that bugs me is, where do you draw the line? Say there's a
> metaprogramming problem at hand. How to decide on solving it with CTFE vs.
> solving it with templates? It would be great to have a simple guideline that
> puts in contrast the pluses and minuses of the two approaches.
>
> It is quite possible that templates get relegated to parameterized functions
> and types, whereas all heavy lifting in metaprogramming should be carried
> with CTFE.

God, I wish we had a real forum with table capabilities. I can't even
rely on monospaced fonts..

Where templates win at metaprogramming:

Templates have pattern-matching capabilities for picking apart types.
CTFE is forced to reimplement part of the D lexer/parser to do so (and
combined with buggy/incompletely specified .stringof, you can't really
depend on your parsing to work right).

With templates, you write "real code", whereas with CTFE you have to
quote everything. D2's token strings improve this somewhat but D1
users are stuck in escaping hell. Not that you care, since you don't
believe D1 even exists anymore, but whatever. Quoting things is still
irritating.

Templates have a definite advantage when it comes to alias parameters
and the instantiation mechanism. CTFE mixins have to worry about
exactly when and where some piece of code is mixed-in, or else you
could be evaluating it somewhere where a given symbol isn't defined.
Oops, you mixed in your code in a different module than the one it was
accessed from, suddenly all the imports are different and you don't
have access to a bunch of functions. With templates, it .. just works.
You can give a template a symbol from some module that it normally
wouldn't have access to and it's perfectly happy. CTFE that deals with
this gets very tedious very quickly.

Templates are much more transparent. Mixin statements/expressions
always look something like a hack. Greppable? Sure. But many times I
don't *want* greppable, I want *clean*.

Meta-metaprogramming. You can't easily write a CTFE map() function.
You *can* write a simple Map template. Reduce!(Cat, Map!(SomeList,
SomeTemplate)) is just.. it brings tears to my eyes. It's so
beautiful.

Where CTFE wins at metaprogramming:

Liiiiiiiiiiists. If you have a list of something, it's far easier to
deal with in an imperative CTFE function than in an awkwardly
recursive template. Edge cases (first, last items) are also easier to
deal with imperatively.

DSLs, and more generally, parsing. Doing DSL parsing with templates is
possible but not fun. You end up with a ton of templates. Not that I'm
advocating parsing lots of things with CTFE either.. you already know
how I feel about that.

Verbosity. Templates require you to add a lot of silly cruft to do
even really simple things. "template SomeListProcessingTemplate(T...)
{ static if(T.length == 0) alias Tuple!() SomeListProcessingTemplate;
else { pragma(msg, T[0].stringof); alias Tuple!(T[0],
SomeListProcessingTemplate!(T[1 .. $])) SomeListProcessingTemplate; }"
Notice the template name is repeated three times within the template..
all the cruft starts to make it difficult to see what's actually going
on. CTFE functions are generally shorter and easier to follow, if for
no other reason than D is mostly an imperative language and that's
what its syntax optimizes for.


Those things being said, please, PLEASE consider __ident. Many times
I'd like to do something with templates because I'd be able to avoid
the CTFE mess with symbol visibility, but I can't because I need to
dynamically generate one stinking identifier from a string. Soooo many
of my CTFE metaprogramming functions could be replaced by simple
templates if I only had __ident. You remember the "finalize" template
thing I wrote once? How it was about 150 lines of hard-to-follow,
terribly hackish CTFE .stringof parsing and bullshit when it could
have been reduced to about 8 lines with __ident? Really. It comes up a
LOT. Consider it.

But one other thing that I want to bring up is that I really, really
hope that this newfound love for CTFE doesn't shift the mindset
towards "why bother adding features to the language when you could
simulate it with CTFE?" For some things, that's fine. But I don't want
to look at D code in 6 or 7 years, see half the source encased in q{},
and see the other half composed almost entirely of mixin(). That's
just awful.