Small Vectors Proposal

[Don Clugston]

Mikola Lysenko wrote:
> Benji Smith wrote:
>> Right now, I'm working on a statistical prediction algorithm that uses 
>> *sparse* 300,000-dimensional feature vectors. There's no reason to 
>> limit SSE optimizations to small-dimensionality vectors.
>>
> 
> I can understand your desire to create a universal solution, and for a 
> long time I too was convinced this was the way to go.  However I have 
> come to understand that there are many practical differences between low 
> and high dimension vectors.  Think about it this way: would you try to 
> use a sparse matrix/vector library for doing operations on 4-d vectors 
> and matrices?  It should be clear that these are two different very 
> different types of problems.

I completely agree. For large vectors and matrices, you're going to have 
to use a loop, and the loop startup costs are amortised; you can use the 
same BLAS code for a 3159 element vector as for a 3158 element one, and 
it makes negligible difference to the speed. But for 3 and 4 element 
vectors it makes an enormous difference.

And the applications are completely different. I imagine games 
programmers don't use much above 4x4 matrices.