Fully dynamic d by opDotExp overloading

[Michel Fortin]

On 2009-04-18 22:21:50 -0400, Andrei Alexandrescu 
<SeeWebsiteForEmail at erdani.org> said:

> I did, but sorry, it doesn't make sense and does nothing but continue 
> the terrible confusion going in this thread.

Then let's try to remove some of that confusion.


> You say: well if opDot were a template then a scripting language can't 
> get to it because what it has is a dynamic string. Yes. That's why 
> opDot does whatever static string processing and then forwards, if the 
> design asks for it, to the dynamic function. opDot's participation is 
> solely a syntax helper within D.

Yes, I understand that template opDot as you proposed is just a syntax helper.

I'm arguing that many uses proposed for this syntax helper are better 
implemented by mixins (because functions added by mixin are accessible 
using reflection, while functions added by a templated syntax helper 
are not). Of the remaining uses that can't be served by mixins, most 
need runtime dispatch, which can be made compatible with a 
runtime-dispatch system if there is a standard name for a non-template 
catch-all function (as opposed to a user-defined name you can only know 
from within a template opDot). And the remaining ones that require a 
template opDot are not really useful (case-insentiveness?).


That was the summary. Following is the detailed explanation. Now, 
Andrei, if you think something doesn't make any sense in all this, 
please tell me where it stops making sense so I can understand were 
we're disagreeing.


Let's consider how runtime reflection works. Basically, you have a list 
of available functions for a given type, and their function pointers. 
When you want to invoke a function using runtime reflection, you find 
it in the list, and call the pointer. Something like this (simplified) 
function:

	void invokeViaReflection(string name)
	{
		void function()* fptr = name in functionList;
		if (fptr)
			ftpr();
		else
			throw Exception("Function "~ name ~" not found.");
	}

(Note that you can already this kind of reflection by building the list 
at compile time for the types you want with something like ClassInfoEx, 
but this could also be built into the compiler.)

So if you have an external script that wants to call a D function, or 
you receive a remote invocation via network, the script will call that 
invokeViaReflection function above which will call the right function.

Now, let's say we want to add the ability for a D class to act as a 
proxy for another object, then we need to catch calls to undefined 
functions and do something with them (most likely forward them to 
someone else). For that to be reflected to the script, we'll have to 
modify the invokeViaReflection like this:

	void invokeViaReflection(string name)
	{
		void function()* fptr = name in functionList;
		if (fptr)
			ftpr();
		else
		{
			void function(string name)* catchAll = "catchAllHandlerFunc" in 
functionList;
			if (catchAll)
				catchAll(name);
			else
				throw Exception("Function "~ name ~" not found.");
		}
	}

The thing is that the name of that "catchAllHandlerFunc" function needs 
to be standardised for it to work with runtime reflection. If the 
standardized way of doing this is to implement things in a template and 
from that template call a catch-all function with a user-defined, 
unpredictable name, then we can't use the catch-all function at all 
with runtime reflection since the name of that runtime catch-all 
function is burried in a template.

This was my first point. I hope it makes sense and that we agree about 
the conclusion: there needs to be a standardized name for a 
non-template catch-all function if we want runtime reflection to 
reflect the compile-time dispatching process.

(By the way this, in itself doesn't preclude having a template 
catch-all too, only if there is one the runtime one should be 
standardized too.)


Now, for my second point, let's consider some use cases for a catch-all 
mecanism.

Let's look at the swizzle case. You want a class or a struct with a 
function for every permutation of "xwyz". Beside doing that manually, 
here's what you can do:

Solution A: use a mixin. This will add 256 functions to your struct, 
and those functions will appear in the functionList used by 
invokeViaReflection.

Solution B.1: use a templated catch-all function, and use the name to 
instanciate the catch-all. Calling the function will work at compile 
time, but at runtime invokeViaReflection is left in the dust.

Solution B.2: in addition to the templated catch-all function in B.1, 
add a runtime catch-all. This is about twice the work, but 
invokeViaReflection can work with those functions.

So of all the solutions above, B.2 is twice the work and adds the risk 
of the two implementations becoming out of sync, and is probably slower 
than A when called through invokeViaReflection. B.1 breaks 
invokeViaReflection. And A works for everything. My conclusion is that 
solution A is preferable. And I think the language should encourage the 
use of solution A in this case.


Now, let's look at the proxy case, where you want to forward function 
calls to another object. In fact, there are two variant of this case: 
one where you know the type at compile-time, one where you don't (you 
only know the base class) but still want to forward all calls.

Solution A: use a mixin. The mixin will create a wrapper function for 
all functions in the wrapped object it can find using compile-time 
reflection, and those functions will appear in the functionList used by 
invokeViaReflection.

Solution B.1: use a templated catch-all function which will simply 
issue a call on the wrapped object. Calling the function will work at 
compile time, but at runtime invokeViaReflection is left in the dust if 
we try to use it on our proxy.

Solution B.2: in addition to the templated catch-all function in B.2, 
add a runtime catch-all that will check the runtime string against the 
function names obtained at compile time and call the right function. 
This is more than twice the work, but invokeViaReflection can call 
functions on our proxy.

Solution C: create a runtime catch-all function that will use 
invokeViaReflection to call the right function on the wrapped object.

Note that solution C is the only option if you don't know the exact 
type at compile-time, like when you only know the base class. 
Unfortunately, solution C for our proxy won't work if the wrapped 
object uses B.1. Same for solution A, which relies on compile-time 
reflection.

So that's why I think B.1 (templated catch-all function) should be 
avoided whenever a mixin solution is possible. In the two use cases 
above, you can use a mixin and get the same "staticness" as in B, but 
the mixin solution also offer support for reflection, which makes it 
superior for the cases you're stuck using the object at runtime, or 
when we want to introspect at compile-time. And when a mixin doesn't 
cut it, the template will most likely just be a stub for calling a 
runtime mecanism.


Note that I'm not that much against a templated opDot, I just want to 
point out its drawbacks for reflection. I'm also unconvinced of its 
usefulness considering you can do the same things better using mixins 
leaving only the runtime case to be solved, where you are better served 
without a template anyway.

-- 
Michel Fortin
michel.fortin at michelf.com
http://michelf.com/