A Perspective on D from game industry

[H. S. Teoh via Digitalmars-d]

On Tue, Jun 17, 2014 at 10:20:59PM +0000, c0de517e via Digitalmars-d wrote:
[...]
> The issue I have with metaprogramming (and overloading and some other
> similar ideas) is that it makes a statement dependent on a lot of
> context, this is tricky in a large team as now just reading a change
> doesn't really tell much. Our is an industry where we still exercise a
> lot of control, we want to know exactly what a statement does in terms
> of how it's executed.

You don't need metaprogramming to have this problem. In my current job,
for example, there is a recent push to move things away from hard-wired
APIs toward more generic APIs, in order to make things more uniform and
easier to use -- instead of memorizing 15 different sets of functions to
use, one for each possible object stored in the database (read_obj,
add_obj, delete_obj, read_file, add_file, delete_file, add_table,
read_table, delete_table, ... ad nauseaum), use a common set of accessor
functions (read, add, delete, update, etc.) under a unified generic
interface. There was also the complaint from some developers that C is
"superior" to C++ because in C++, a method call in a complex class
hierarchy can theoretically end up "anywhere", whereas in C, you at
least know exactly which function will get called, since there is no
overloading.

Well guess what? In order to implement the unified generic interface, we
ended up with tables of function pointers that get passed around, and
now you have code that looks like this:

	/* Note: this is not the real code, I just made it up to
	 * illustrate the problem */
	int my_generic_func(generic_container *container) {
		if (container->ops.find("some_key")) {
			generic_item *item = container->ops.read("some_key");
			value_type *value = item->ops.read("some_field");
			item->ops.add("new_field", &value);
			return 1;
		} else {
			generic_item *item = container->ops.make_new_item("new_data");
			container->ops.add("new_key", item);
			return 0;
		}
	}

Now you discover a bug somewhere in this function. How would you go
about finding where it is? Well, since this is C, which allegedly
doesn't have any polymorphism, that should be trivial, right? Just trace
through all of the function calls and narrow it down. Except... all of
those ops.xxx() calls are function pointers. So OK, we need to find out
where they point to. But they are set in the *container struct somewhere
far, far away from this function, so how do we trace them? Now we have
to search through the entire source tree to find every place where
container structs have their function pointers set. Except... there are
several different kinds of containers, and they all have radically
different implementations of each function. So how do we know exactly
which container type got passed in? Worse yet, even if you manage to
narrow that down, you still can't resolve where item->ops.add points to,
because containers may contain different kinds of items, and what type
of item gets put in there is only known at runtime... Aaargh...

If C, which purportedly is unambiguous as to exactly what each statement
does -- there is no polymorphism (allegedly), no function overloading,
no operator overloading, etc., still exhibits exactly the same context
dependence that you object against, then I'm forced to conclude that
this context dependence is a red herring in your argument against
metaprogramming.


> I won't replicate what I wrote on the blog here, so if you're
> interested I'd love to have more comments on that aspect, but that is
> why I care about productivity but I'd rather prefer to gain that with
> faster iteration and language features that don't make semantics more
> flexible, than metaprogramming.
> 
> Then of course a tool is a tool and I'd always love to have -more-
> tools, so I'm not saying metaprogramming is a bad thing to have. Like
> OO is not really a bad thing to have either. But there are tools to be
> used with certain care. Metaprogramming -mentality- is scary to me
> like OO-heavy thinking is.

Believe me, I totally sympathize with where you're coming from --
especially after having to deal with C code like I illustrated above,
where every other line is a call to a function pointer that points who
knows where! It used to be, in the supposedly bad ole days of having
hundreds of similarly-named functions (add_file, add_table, add_obj,
delete_file, delete_table, delete_obj, etc.), that I can just run ctags
and use vim's tagging function to follow function calls with a single
keystroke, and I can rest reasonably assured that it will take me to the
function in question, and that it represents the sequence of execution
at runtime. Nowadays, tagging is basically useless, because every other
line calls container->ops.add() or container->ops.delete(), and there
are 15 different implementations of add() and delete(), who knows which
one it ends up calling at runtime?!

OTOH, OO is a huge time saver, when used correctly, because it allows
you to reason in the abstract instead of losing sight of the forest for
the trees of details you have to wade through, just to do one single
simple task. It reduces cognitive load: instead of memorizing the names
of add_obj, add_file, add_table, delete_obj, delete_file, etc., you only
need to remember add and delete, and the abstraction takes care of
itself by resolving to the correct overload based on the argument types.

Metaprogramming goes one step further and lets you reduce boilerplate --
which is the source of subtle bugs caused by typos, not to mention a
maintenance nightmare when fixing a bug in one instance of boilerplate
doesn't fix all other instances of the same bug in the same boilerplate
that's copied over 50 other places in your code -- while presenting an
easy to remember abstract interface that you can remember in your sleep.
In fact, it allows you to centralize several different implementations
of the same logical operation under a single function, so even if you
don't necessarily know exactly which version of the function will get
called at runtime, you still know exactly what the code looks like, 'cos
they all come from the same template. :)

At the end of the day, *all* tools must be used with care. Even in C, a
language with neither OO nor metaprogramming support, you can code
yourself into a nasty mess by using built-in language constructs like
function pointers, like I showed above. Just because you *can* cause
serious injury to yourself with a hammer, doesn't mean that hammers
are inherently evil.


T

-- 
My program has no bugs! Only undocumented features...