SciD: the humble beginning

[Lars T. Kyllingstad]

dsimcha wrote:
> == Quote from Lars T. Kyllingstad (public at kyllingen.NOSPAMnet)'s article
>> I've just created a new dsource project and uploaded my collection of
>> numerical routines. You can find it here:
>>    http://www.dsource.org/projects/scid
>> Hopefully others will find it useful as well. Most of the code is
>> written with a "get it to work - NOW" philoshophy, and as such there is
>> a lot of potential for improvement and polish, not least when it comes
>> to performance. I am aware of this, so there is really no need to
>> benchmark it against established scientific libraries quite yet. ;) More
>> routines will be added as I write them (which is usually when I need
>> them for work), and the existing ones will be polished whenever I have
>> time to do so.
>> Huge thanks to Bill Baxter for letting me use his BLAS and LAPACK bindings!
>> Requirements:
>>    DMD 2.037
>>    BLAS and LAPACK libraries
>> -Lars
> 
> This looks absolutely terrific given that it's a work in progress and all.  I
> agree that we shouldn't be too obsessed with performance yet because:
> 
> 1.  **Make it work**, make it right, make it fast.  We can always optimize it
> later provided that there aren't any ridiculously bad design decisions.

A lot of the code is ported from 20-30 year old (i.e. extremely well 
tested) FORTRAN code, so any performance gains would be a bonus provided 
by the D language itself, and most likely not in the algorithms.

Also, if you spot any ridiculously bad design decisions, let me know. ;)


> 2.  At least some of the time people (myself included) would rather a nice API and
> decent performance than an ugly API (read:  a straight Fortran or C API) and
> lightening quick performance.

Yeah, that's what I think too. And, the way it is now, people who for 
some reason want to use the old-style api can just access the scid.ports 
functions directly.


> We've got the beginnings of a good matrix/numerics lib in your work.  I believe
> that my dstats lib would provide just about all the statistical functionality a
> good scientific lib would need, and I'm getting close to declaring it
> beta-quality.  I'd say based on a quick look at what scipy has that we also need
> the following for a credible full-fledged scientific lib:
> 
> Machine learning, i.e. classification and clustering.  (I'd probably be qualified
> to write that and was halfway thinking of breaking ground on it over Christmas
> break.  However, I'm not committing to this yet, so if someone else already has a
> work in progress, let me know.  Also, there's a Java machine learning library
> called Weka that I've used before.  It's very complete but the API is painful and
> it's GPL, so IDK if it'd be worth it to port to D.)
> 
> Optimization.  (A few people have tried but IDK if they've actually gotten far off
> the ground with it.)
> 
> Basic image I/O and processing.
> 
> Plotting.  I've considered doing this a few times, but I've decided it needs to be
> put off until D2 is stable and the GUI toolkits for it are reasonably stable.  One
> layer of instability (D2 itself) is workable, but two layers (D2 and the GUI libs)
> is not.

I agree with all of this. Here's a piece of advice to anyone who wants 
to try their hand at these and other things: There are a LOT of 
established and *extremely* well-tested numerical libraries lying around 
on the internet, and many of them are in the public domain. Instead of 
spending a lot of time re-inventing the wheel (I've tried it), port some 
old code and try to improve on it when you get it working. Here are a 
few starting points:

   http://www.netlib.org/
   http://gams.nist.gov/

And I forgot to say in my original announcement: I welcome contributors 
with open arms. I have no intention of making this a one-man show. If 
you want to contribute or participate in some way, let me know.

-Lars