The Matrix to end all Matrix classes (Let's dream!)

[Oskar Linde]

Don Clugston wrote:
> No, I don't have the perfect Matrix. I just want to explore the question 
> of whether it exists.

A very interesting topic indeed. I will just give some quick thoughts to 
start with. First of all, a matrix is (generally) 2-dimensional, but 
there are many uses for higher-dimensional structures as well. Also, the 
1 dimensional vector is a useful structure that could be handled in the 
same generic framework. Is the target just 2-dimensional matrices or 
also higher dimensional arrays?

The matrix as a mathematical structure is quite different in semantics 
and use from generic multi-dimensional arrays and should perhaps be 
treated separately.

To avoid confusion, I will use "size" to refer to the number of elements 
of a matrix/array and "dimensionality" to refer to the number of 
dimensions of an array (a vector is 1-dimensional, a matrix is 
2-dimensional).

First, there is the separation of compile time and run time. One can 
identify (at least) 3 different needs, with different requirements for 
run-time/compile-time dynamics:

* Small static matrices/arrays (where the size and dimensionality are 
known at compile time)
* Dynamic matrices/arrays (where the sizes are determined at run-time, 
but dimensionality is fixed at compile time)
* Fully dynamic multidimensional arrays (where the dimensionality is 
determined at run time)

The third case could probably in most cases be left out, but the first 
two make an interesting separation. Knowing the size at compile time is 
mostly an advantage for smaller matrices (eg 4x4, as commonly used for 
geometric translations). Larger matrices (eg 1000x1000) probably gain 
much less from compile time known sizes. There is also the matter of 
template bloat that means that compile time quantities are best left to 
the smaller numbers.

> Requirements:
> 1. Must be compatible with future multi-dimensional array syntax such as
> double [7,5] m;
> 2. Must be theoretically capable of optimal performance.

Seems reasonable.

> 3. Needs to support slices. This means we also need to support strided
> vectors. Therefore we also need matrix slices, and matrix references.

Yes, both strided views and non-strided references are definitely needed.

> Probably distinguishing between the T[] and T[new] concepts.

Such a distinction is definitely of an advantage. For large matrices, 
ownership of memory becomes important, as such structures may be too 
large to be able to rely on the GC.

> 4. For storage we need packed, triangular, symmetric, banded, and sparse
> matrix support. (Possibly even 'calculated' matrix support, for
> user-defined matrix types which generate entries on-demand).

For generality, at least sparse (and possibly calculated) are useful for 
vectors and higher dimensional arrays too.

> 5. Can be generalized to tensors.
> 6. Nice error messages.
> 
> I now have enough experience with my BLADE library to be confident that 
> all of these features can be achieved in existing D. In particular, the 
> crucial requirement 2 can be completely decoupled from the others.

Sounds excellent.

> But, there are many requirements (dozens?) which are not on my brief 
> list, and perhaps they cannot all be met simultaneously, even in theory. 
> Yet I'm not convinced that it's impossible. What are the other 
> requirements?

Naturally, different element types (as well UDT). Compatibilty with 
external libraries (Fortran/C) with the possibility of switching between 
Fortran and C layout.

-- 
Oskar