Scala future, Sing#

[bearophile]

Jari-Matti M.:

>There are certainly truckloads of features that could be taken from Scala to D.<

But D2 is already quite complex, so it's better to add things carefully. For example patterm matching is useful, but it adds a lot of complexity too.


>But I'm afraid that the desire to have familiarity and compatibility with the C/C++ family is more important in this community than cool new functional features.<

And this can be a good thing, because C and C++ are commonly used languages.


>- community: From what I've gathered, the Scala community mostly consists of
much more experienced computer scientists and programmers<

This is an advantage for D: it can be used by more ignorant people too (or people more ignorant of functional lanmguages).


>- bloat: Scala is more lightweight.<

This is a matter of balance, and there are no 'best' solutions. Moving things from the library to the language has some advantages.


>Several optimizations that improve the performance of HOFs are already known, but the compiler and virtual machine are not yet as good as they can be. In theory a good supercompiler can make Scala as fast as D.<

HotSpot Java GC is much more efficient than the current D GC, and HotSpot is often able to inline virtual methods. D has the advantage of having a simpler core. Creating a Scala compiler on LLVM may be hard, running D1 on LLVM was easy enough. Simpler systems have some advantages. Scala type inference is much more powerful but it's also harder to use (if you want to do complex things), and requires a more complex compiler.
In practice supercompilers are very hard to create, while D1 code running on the LLVM is already very efficient.
Simpler systems also have the advantahe of being more transparent: understanding why some D1 code is fast or slow is probably simpler than doing the same thing with a piece of Scala code.


>I personally find it funny that the meta-programming features in D are perfect for shrinking the language core, but every year new features still keep creeping it.<

They are not perfect, they have limits, and the ersults aren't always nice, see the struct bitfields.


>- dynamics: Scala is more dynamic (reflection, class loaders etc.) Thanks to JVM.<

Some of such things can be added/improved in D too.


>- syntax: Scala has a consistent and orthogonal syntax.<

Too much orthogonality is bad, it produces the LEGO disease. A compromise is better.


>Syntactic sugar is used sparingly and when it's being used, it shaves off boilerplate quite a bit.   * e.g. (_._2 * _._1)<

Here there are risks too. I have seen a syntax for a fold (reduce) in Scala that's horribly unreadable. Python3 has even removed reduce() from the core language because folds aren't easy to understand and I agree with their decision. Keeping the language easy is more important. Too much boilerplate is boring, but the boilerplate is better than hard to understand code.


>* In D syntactic sugar often saves only a couple of characters (like the new template T!X syntax)<

I agree that was a silly idea, that I may even like to remove from D2. I want eager/lazy sequence comphrensions :-)


>- memory management: the JVM's GC is pretty mature, but of course manual memory management isn't as easy as in D<

Some forms of memory usage that I use in D are impossible on the JavaVM.


>- compatibility: D wins (?) here if C/C++ compatibility is important, but Scala is compatible with the large existing code base of Java, though<

D refuses to be compatible with C++. There's just partial compatibility (and this is probably good).


>the huge amount of bugs made it unusable for me.<

There are many bugs in D, but D is slowly opening a bit more and more toward the community, for example see the recent lot of bug fixes by Don. If more people will work like Don, lot of bugs will be removed. Walter is slowly understanding what open source development means. So I have hope still.


>And I'm also so happy to find that thanks to authors' knowledge of type systems (dependent types, HM, System F, etc.) Scala is a genius at inferring types. D doesn't really have a clue.<

Scala uses a totally different type system. I think it uses an Hindley–Milner type inference algorithm. Walter is probably not expert on such thing. A single person can't be expert on everything. Today designing concurrency, type inference or garbage collectors requires lot of specialized and even some academic knowledge. Scala author has used the JavaVM to avoid doing lot of low-level work. D type system isn't so bad. It has limits, and some of such limits may be lifted a little, but you can do lot of things with the D language anyway, see some of the things I've done in my dlibs. Keeping the type system simpler has some advantages too, for example compilation speed.


>Especially the array literal type inference is really naive.<

I'm sure it's not hard to fix the array literal type inference that currently is not good; it's just that Walter isn't interested in doing it, or he thinks things are good like this, like for the half-unfinished module system.


>- to summarize: I use Scala for high level tasks, and came back to D when I need to see the actual machine code and optimize some tight inner loop. D is sometimes more suitable for this than C/C++ since it has a bit saner syntax and high level abstractions.<

Today when you need hi-performance you need the GPU or to use SSE registers very well.
An usage usample of the GPU:
http://www.smartwikisearch.com/algorithm.html

In the end if you need really fast programs that have to perform heavy numerical computations you need languages like Python (plus the right libs, like CorePy), D (and Scala) isn't up to the task yet:
http://www.corepy.org/
http://mathema.tician.de/software/pycuda
http://python.sys-con.com/node/926439
http://pypi.python.org/pypi/python-opencl/0.2

Bye,
bearophile