Scala Spark-like RDD for D?
data pulverizer via Digitalmars-d-learn
digitalmars-d-learn at puremagic.com
Mon Feb 15 03:40:34 PST 2016
On Monday, 15 February 2016 at 11:09:10 UTC, data pulverizer
wrote:
> Are there are any plans to create a scala spark-like RDD class
> for D
> (https://www.cs.berkeley.edu/~matei/papers/2012/nsdi_spark.pdf)? This is a powerful model that has taken the data science world by storm; it would be useful to have something like this in the D world. Most of the algorithms in statistics/data science are iterative in nature which fits well with this kind of data model.
>
> I read through the Kind Of Container thread which has some
> relationship with this issue
> (https://forum.dlang.org/thread/n07rh8$dmb$1@digitalmars.com).
> It looks like Immutability would be the way to go for an RDD
> data structure. But I am not wedded to any model as long as we
> can have something that performs the same functionality as the
> RDD.
>
> As an alternative are there plans for parallel/cluster
> computing frameworks for D?
>
> Apologies if I am kicking a hornet's nest. It is not my
> intention.
>
> Thanks
Perhaps the question is too prescriptive. Another way is: Does D
have a big data strategy? But I tried to anchor it to some
currently functioning framework which is why I suggested RDD.
More information about the Digitalmars-d-learn
mailing list