Bartosz Milewski Missing post

[Bartosz Milewski]

Andrei Alexandrescu Wrote:

> Scala doesn't know what to do about threads. 

That's my impression too, although Scala's support for actors leaves D in the dust.

> The trend I'm seeing is 
> that functional languages are getting increasing attention, and that's 
> exactly because they never share mutable memory. As far as I can see, 
> languages that are based on heavy shared mutation are pretty much dead 
> in the water. We have the chance to not be so.
> 

It's a very sweeping statement. I just looked at the TIOBE index and couldn't find _any_ functional languages in the top 20. 

> I'm 
> not at all sure the focus must be put on high-level race avoidance, 
> particularly given that the cost in perceived complexity is this high.
 
The complexity argument is tenuous. It might look like my proposal is complex because I'm dropping the whole system in at once. But it's a solid, well thought-out system. What I see happening in D is the creeping complexity resulting from sloppy design. We've been talking for years about closing some gaping holes in the design of arrays, slices, immutable, qualifier polymorphism--the list goes on--and there's little progress. There is no solid semantics for scope and shared. A solid solution to those issues will look complex too. 

> Why wouldn't we think of making message passing a language feature in D? 

Because we don't have even a tiniest proposal describing it, not to mention a design.

> Why does message passing need erlangization to support message passing?

Because the strength of the Erlang model is the isolation of processes. Take away isolation and it's no better than Scala or Java. Granted, having to explicitly mark objects for sharing in D is a big help. Here we agree.

> What I discussed was a holistic approach in which language + standard 
> library provides a trusted computing base. 

Have you thought about how to eliminate data races in your holistic approach? Will "shared" be forbidden in SafeD? Will library-based message-passing channels (and actors?) only accept simple value types and immutables?

Andrei, you are the grand wizard of squeezing powerful abstractions out of a kludge of a language that is C++ with its ad-hoc support for generics. It's impressive and very useful, but it's also hermetic. By contrast, generic programming in D is relatively easy because of the right kind of support built into the language (compile-time interpreter). 

I trust that you could squeeze powerful multithreading abstraction out of D, even if the language/type system doesn't offer much of a support. But it will be hermetic. Prove me wrong by implementing a message queue using the current D2 (plus some things that are still in the pipeline).

> How is this related to the discussion at hand? You want to put all 
> concurrency support in the language. That is, you want to put enough 
> power into the language to be able to typecheck a variety of concurrent 
> programming primitives and patterns. This approach is blindsided to the 
> opportunity of defining some of these primitives in the standard 
> library, in unsafe/unportable D, yet offering safe primitives to the 
> user. In the process, the user is not hurt because she still has access 
> to the primitives. What she can't do is define their own primitives in 
> safe D. But I think that's as useless a pursuit as adding keywords to C 
> to allow one to implement malloc() in safe C.
> 

That's a bad analogy. I'm proposing the tightening of the type system, not the implementation of weak atomics. A better analogy would be adding immutable/const to the language. Except that I don't think const-correctness is as important as the safety of shared-memory concurrency.

> > Without
> > type-system support, though, such queues would have to be either
> > unsafe (no guarantee that the client doesn't have unprotected aliases
> > to messages), or restrict messages to very simple data structures
> > (pass-by-value and maybe some very clever unique pointer/array
> > template). The latter approach introduces huge complexity into the
> > library, essentially making user-defined extensions impossible
> > (unless the user's name is Andrei ;-) ).
> 
> Complexity will be somewhere. The problem is, you want to put much of it 
> in the language, and that will hit the language user too. 

I'm being very careful not to hit the language user. You might have noticed that my primitive channel, the MVar, is less complex than a D2 implementation (it doesn't require "synchronized"). And the compiler will immediately tell you if you use it in an unsafe way.

> Semantic 
> checking will always be harder on everyone than a human being who sits 
> down and implements a provably safe library in ways that the compiler 
> can't prove.
> 

Be careful with such arguments. Somebody might use them to discredit immutability.

> > Let's not forget that right now D is _designed_ to support
> > shared-memory programming. Every D object has a lock, it supports
> > synchronized methods, the keyword "shared" is being introduced, etc.
> > It doesn't look like D is moving away from shared memory. It looks
> > more like it's adding some window dressing to the pre-existing mess
> > and bidding its time. I haven't seen a comprehensive plan for D to
> > tackle concurrency and I'm afraid that if D doesn't take a solid
> > stance right now, it will miss the train.
> 
> I think we can safely ditch this argument. Walter had no idea what to do 
> about threads when he defined D, so he put whatever he saw and 
> understood in Java. He'd be the first to say that - actually, he 
> wouldn't tell me so, he _told_ me so.
> 

I realize that. Except that the "shared" concept was added very recently. 

> To me, adding concurrency capabilities to D is nothing like adding 
> window dressing on top of whatever crap is there. Java and C++ are in 
> trouble, and doing what they do doesn't strike me as a good bet. 

So far D has been doing exactly what Java and C++ are doing. My proposal goes way beyond that. But if you mean D should not support shared-memory concurrency or give it only lip service, than you really have to come up with something revolutionary to take its place. This would obviously not make it into D2 or into your book. So essentially D2 would doomed in the concurrency department.

> You're 
> right about missing the train, but I think you and I are talking about 
> different trains. I don't want to embark on the steam-powered train.

Should we embark on a vapor-powered train then ;-)