Scala Spark-like RDD for D?

data pulverizer via Digitalmars-d-learn digitalmars-d-learn at puremagic.com
Mon Feb 15 03:40:34 PST 2016


On Monday, 15 February 2016 at 11:09:10 UTC, data pulverizer 
wrote:
> Are there are any plans to create a scala spark-like RDD class 
> for D 
> (https://www.cs.berkeley.edu/~matei/papers/2012/nsdi_spark.pdf)? This is a powerful model that has taken the data science world by storm; it would be useful to have something like this in the D world. Most of the algorithms in statistics/data science are iterative in nature which fits well with this kind of data model.
>
> I read through the Kind Of Container thread which has some 
> relationship with this issue 
> (https://forum.dlang.org/thread/n07rh8$dmb$1@digitalmars.com). 
> It looks like Immutability would be the way to go for an RDD 
> data structure. But I am not wedded to any model as long as we 
> can have something that performs the same functionality as the 
> RDD.
>
> As an alternative are there plans for parallel/cluster 
> computing frameworks for D?
>
> Apologies if I am kicking a hornet's nest. It is not my 
> intention.
>
> Thanks

Perhaps the question is too prescriptive. Another way is: Does D 
have a big data strategy? But I tried to anchor it to some 
currently functioning framework which is why I suggested RDD.



More information about the Digitalmars-d-learn mailing list