Is metaprogramming useful?

[Georg Wrede]

Brad Anderson wrote:
> Jarrett Billingsley wrote:
> 
>>A little off the topic, but not really, as it spins off of the idea of a 
>>compiler as an interpreter.  I always thought it would be an interesting 
>>exercise to make a language where metaprogramming is not only possible, but 
>>nearly as full-featured as the actual code.  That is, templates (or whatever 
>>they'd be called, because they'd be far more advanced) would be a script for 
>>the compiler, which could even be compiled to bytecode.  The input would be 
>>language constructs -- symbols, statements, expressions -- and the output 
>>would be code which could then be compiled.  It'd basically be a scriptable 
>>compiler.

You've just described Lisp. And very precisely, at that.

And today's Common Lisp implementations blend compiler and interpreter 
so thoroughly that you don't know which is going on when.

> Poor Lisp.  It just sits there, 50 years old, debugged, optimized, and ready
> to go, while the imperative languages try to inch closer over the decades.

Somehow I've always felt that it's mostly not about other languages copy 
catting features. In Lisp's case it's more like, there's a Right Way to 
do things, and Lisp has found many of them right from the start.

> So OOP comes along.  Lisp adds it (in a superior way, imo)
> So AOP is (or will be) hot.  Lisp adds it.
> Metaprogramming?  The MetaObject Protocol appears, to complement the already
> amazing macro facility.

The most amazing thing about Lisp is how simply trivial all the hard 
stuff is. So, someone invents OOP, no problem, let's write a complete 
OOP framework, in just 200 lines of code. AOP anyone, hey, that'll be 
ready by tomorrow morning.

And Metaprogramming? Ha, even the trivial (3kloc, in D) Lisp-in-D 
implementation (dLISP) on dsource has defmacro. (Which actually works!)

> I'm not knocking the imperative languages, as a lot of people know them and
> use them successfully.  I am more amazed at how such a valuable toolset is
> consistently under-used and its functionality is rewritten from scratch.

There's a single reason for it. Lisp is /scary/. It's scary in itself, 
but even worse, folks get scared that it undresses them. ("Start mucking 
with Lisp, and they might see that you're not as smart as you've led 
everyone to believe.") And the parentheses are simply a convenient 
excuse for abstinence.

> As this thread talks about metaprogramming and syntax, the elegance of Lisp's
> code being the same as its data is relevant.  To be the ultimate expression of
> metaprogramming in this way, you cannot do it in a new language.  You'd end up
> merely with another flavor/dialect of Lisp.

Well, Lisp is the attractor towards which the D meta language is moving, 
whether we want it, or not. We've even moved quite fast lately. And, 
we're actually much nearer to Lisp now than most folks here even suspect!

> Greenspun's 10th Rule of Programming:  Any sufficiently complicated C or
> Fortran program contains an ad hoc, informally-specified, bug-ridden, slow
> implementation of half of Common Lisp.

It's not a coincidence that I installed Allegro Common Lisp (from Franz 
Inc, a demo version of an excellent commercial implementation) on my 
Fedora last week. And dLISP for comparison.