GSoC 2012 Proposal: Continued Work on a D Linear Algebra library (SciD - std.linalg)

[Jens Mueller]

Cristi Cobzarenco wrote:
> On 24 April 2012 08:32, Jens Mueller <jens.k.mueller at gmx.de> wrote:
> 
> > Cristi Cobzarenco wrote:
> > > Unfortunately, my proposal was not picked up this year. I might try to
> > work
> > > on these ideas this summer anyway so I would still be very much
> > interested
> > > in ideas and feedback, but I will probably have much less time if I'll be
> > > working somewhere else.
> >
> > I'm less familiar with your SciD code base but I have used Eigen
> > regularly. Maybe you can answer my questions right away:
> > 1. How are expression evaluated? Eigen uses no BLAS backend. All
> >   code is generated by themselves.
> >   Do you plan for such an option?
> >
> 
> The expressions are built in a similar way to how they are in Eigen (any
> many other linear algebra libraries), using expression templates that
> build, essentially, abstract syntax trees which get evaluated on assignment
> (or with the eval() function). The back-end for evaluation can be specified
> using a version flag. Specifying "version=nodeps" results in code being
> generated by the library with no external dependencies - this is much
> slower, as it doesn't use SIMD operations or anything of the sort.
> 
> In the current state of the library, this is done by essentially providing
> naive implementations of the BLAS functions. In the revamped version, I
> plan this to be done in slightly different way (more similar to the way
> Eigen works) which removes the need for temporaries in some cases where
> using BLAS/LAPACK makes temporary allocation inevitable. In the immediate
> future I do not plan to include SIMD operations for the version=nodeps
> version and the library will be the most efficient when using BLAS & LAPACK.

I see. That means you have already an expression evaluator. I still
wonder why a[] = b[] + c[] shouldn't be handled completely by the
compiler. I thought that this is sufficient for the compiler for
vectorization.

<snip>

> > 2.2. Are there places where a D solution may improved over the Eigen?
> 
> I'm not entirely sure. There's the obvious stuff like more natural slicing
> syntax which D allows us to do.  Easily swappable backends is another thing
> - I'm writing everything to allow custom backends for the matrices (the
> backend will be a template parameter in the new version). This would in
> theory, allow GPU matrices to be easily written if anyone
> wanted (particularly if someone wrapped something like CUBLAS).

Sounds interesting.

> I am actually very interested in hearing from other people if they think
> there's stuff which we could do better thanks to D.
> 
> 
> > 3. I'm not so sure about the array() stuff. I never liked in Eigen.
> >   Being able to call std.algorithm on the raw memory may be sufficient
> >   for the time being.
> >
> 
> I am starting to lean against array myself. I wanted mostly to allow for
> arbitrary typed, higher-dimensional matrices with element-wise operations.
> But having worked a bit on the changes already, I realised that it's very
> easy to disallow certain matrix operations on types which do not support
> multiplication for example. Also I think I found an elegant way of
> supporting higher-dimensional matrices using the same type, but I have to
> do a bit more research into that.

Nice.

> > 4. How about a sparse storage backend for sparse matrices? I'm missing
> >   sparse matrices in Eigen even though the situation is improving but
> >   they're not fully integrated yet.
> >
> 
> Definitely. Sparse storage is a long term goal for the library, but not in
> the first iteration.

Don't want it in the first iteration. Just want to make sure that
current design allows a straightforward extension in that direction.

> > I'd like to support your efforts for including your work in Phobos. How
> > about you clone Phobos and gradually move your work into a work in
> > progress branch? Really little steps. This allows me to follow you
> > closely. To familiarize myself with the code and I would then fill in
> > unittest, documentation and benchmarking code where missing as we go.
> >
> 
> That is actually a better idea than what I had in mind. I will do that. I
> am really glad you're interested in the project, but unfortunately I won't
> be able to start work on the library until the beginning of June. Also the
> proposal for the project was meant as my full-time job this summer, but
> since I'll probably be working somewhere else now you should take into
> account that I probably won't be able to stick to the schedule I gave in
> the proposal.

Take your time. I need to familiarize myself with the code anyway.

Jens