Dispatching on a variant

[Justin Johansson]

Jarrett Billingsley Wrote:

> On Sat, Sep 26, 2009 at 10:36 AM, Justin Johansson
> <procode at adam-dott-com.au> wrote:
> 
> > I've got about 2 dozen types in the variant so the O(n) really hurts.
> > The variant thing seemed like a really cool idea at the time but now ...
> > Without something like suggested above or a computed goto on typeid or Andrei's visitator,
> > it almost pushes me to backout from using variants and having to redesign around some common base class or interface and using virtual function dispatch. :-(
> 
> Ouch. 2 dozen types? No offense, but that sounds like "you're doing it
> wrong." Revising your strategy is probably for the best.
> 
> What are you doing, anyway?


Sorry I can't go too much into the confidential details of this
particular project.  However if you remember when I joined this
group, what is it, 3 weeks ago I gave a short bio of myself by
way of introduction.

So knowing my background in electrical engineering, maths & physics
and being a C++ hacker of 20 or so years, you might well guess
that my specialist domain is in industrial control and scientific
data analysis.  You might say its about real-time, high-speed
gaming (discrete and analog sensor inputs instead of joy sticks)
but without any GUI :-)

There is currently a desire to fit a scripting layer over a highly
tuned core application (written in C/C++) that works with
strongly-typed, mostly numeric value-based data.  Typing in this
sense largely means range-checked. So, picking a simple example,
an integer in the range [0..999] is a different data type to another
in the range say [-4000..4000].  Most of the datatypes (size-wise)
fit conveniently into a union of (short, int, float, double), but,
it's the value space of the data that determines its type.

The primitive numeric types are typedef'ed in C++ code. So you might
have, eg, typedef int16 sensor1_t; representing the range checked value
[0..999] from sensor type 1.  All in all there's about 18 different
primitive data types. (btw. this explains the reason for asking the
"is typedef an alien in D" question on this forum a few days ago).

More complex datatypes in this system are composed as tuples of the
"primitive" datatypes in the system-wide data dictionary.  Accordingly
there's a lot of variable-width data and this is demanding on heap
allocation.

So that's the background in vanilla terms.  The scripting layer is
essentially a DSL.  It's been decided to use a strongly-typed FP
paradigm for this exercise and with a leaning towards immutable data.

Hopefully this design decision will make for easier system T&A (test
and acceptance).

Now looking into different language options for constructing this
DSL, no question about it; the implementing language must support GC.

So far I've considered Java and Scala but there are performance-
crippling JNI issues.  Also given C language interoperability plus GC,
right now D is looking like a much better bet.

Besides, aside some compiler bug issues, D looks like a lot more fun :-)


Getting back to core business, both memory and speed criteria are
important for this project.  I'm hoping to use primitive data types,
rather than wrapping low-level data in an object, to get a performance
benefit.

Up until now, Andrei's discriminated union (aka variant) was looking
like a good idea, using probably about half the memory(***) compared with
the everything-is-an-object approach in Java.  For my application, I'd
be using the general Algebraic form rather than the standard Variant to
model my 18 primitive data types.

*** btw. Here's some miscellaneous links wrt Java object overhead:

http://www.javamex.com/tutorials/memory/object_memory_usage.shtml

http://devblog.streamy.com/2009/07/24/determine-size-of-java-object-class/

http://www.javaspecialists.eu/archive/Issue142.html


At the end of the day, if I can get this thing to fly using D, it will,
at least to me, prove that D is a killer language.


Thanks for listening.

-- Justin Johansson