Revamped concurrency API

[bearophile]

Bartosz Milewski:

>It's good to know that my ideas are still circulating in the D community. Thanks, guys!<

They will keep circulating! And there's not just D in computer science, future computer languages may use your ideas. Someday I'll design my own C-like language, simpler (and more limited in scope) than D ;-)


>Since I'm now put on the spot, I have to explain my recent detachment from D.<

This makes me (and surely many other persons here) a little sad. 

D is an engineering project, and it's not small. And the first thing to learn about all sizeable real-world engineering projects (like the creation of a large bridge, or many other things) is that they are first of all politics. They are not the result of rationality, even if often some rationality is required. All large engineering project come out of a large amount of compromises. If you pretend to see one of such systems to come out of rationality alone, you will never see a real one. 

So every time you have to fight for the rationality to prevail, and you have to be ready to accept some compromises. It's life.


>it's hard for me to subscribe to the "good enough" philosophy (as long as it's better that C++, it's fine for D).<

D language is largely grown, and not designed. D is not Scala or Haskell. I think there is not enough brain mass in this group to design rationally a C++-class language. Probably there's not even brain mass to design a Python-class language. D2 will surely have a big amount of design mistakes, every one here can list you some of them. But one of the best qualities of Walter is that he's not frozen, he sometimes seems to move very slowly, but he never stops moving, and usually he eventually sees the light (and when this happens it's usually a small surprise for everyone else, like the show of a rabbit pulled out of a hat). So I think some of the design mistakes of D2 will be fixed in D3 :-)


>My impression is that, as the release of D2 and the publication of Andrei's book are nearing, this attitude is gaining ground. I try to fight this attitude but it's an uphill battle. Or, as Andrei puts it, I start whining and give up ;-).<

Andrei's book comes too much early, forcing rhythms that I think aren't good for D2. On the other hand no software is ever finished, it's just abandoned. And you must not think of D2 as the finished product :-) You may even think of D2 as just a bridge toward D3 :-) D2 is probably just an experiment, to experimentally see what design ideas are good and what to throw away and redesign. And such things are slow. So you may want to come back in few years, to try to improve the concurrency design of D3 :-)


>The semantics of "shared." I can live with postponing the implementation of the race-free type system, but not with the compiler inserting barriers around all shared reads and writes, even inside synchronized sections.<

This is a topic that can be discussed. If enough people think this is a problem, then eventually Walter may change his mind, or he may design something better, etc. If no one discussed problems they can't be fixed/improved. I know that discussing things and accepting compromises is a pain, but it's an inevitable pain. Rationality alone usually never wins, you need politics too.


>The C++-derived template metaprogramming mess. Especially when C++0x provides better support for variadic templates than D (see my upcoming blog). I fought successfully for non-functional approach to string mixins. The same is needed for more general templates. In my opinion, there should not be any part of the language that is understandable only by seasoned library writers.<

Again, this seems a small piece of D2, so it may be fixed if enough people think this is bad. You can explain what do you want to change here, and people will see and think. Eventually it may even be fixed.


>The "better than Java" aspect of D ignores the latest development in Java. In particular the work on non-null pointers<

We have discussed weeks about nonnull references in D (and I have even discussed about nonnull pointers too), I have done my best. A good percentage of people have said that they like to try this idea. Walter in the end was not interested (I think). But maybe in future the situation will change. There's always hope on this. At the end I have seen no one say "This is a totally wrong idea and will never see the light in D".


>and the use of annotations for type-system extensions. Annotations are being added to D as we speak because it finally became obvious that no amount of cleverness can make object properties work without additional syntax. So the patching begins, without a clear vision of the role of annotation in future D.<

Annotations in D are just an embryo, and I think they come from no design yet, they are just a bit copied from Java. If you have a vision for their role in D, then it's a very good moment to talk and show such vision. I am sure we will be interested to listen. It's surely not too much late to design them well.


>As far as my thread work went, I had an almost working implementation of spawn, except for a nasty compiler bug which resisted all efforts to reduce it to a simple test case. Threads also required some clever work with templates (testing arguments to "spawn" for sharing violations).<

Walter isn't magic. If some feature is very hard to implement then it may be better to not add it until a good implementation is actually created.


>I implemented typelist.d in Phobos to make it easier, and learned how inflexible D templates were. At least they are better than C++, or are they?<

D templates are not "better" than C++ ones, they are simpler to use, less bug-prone, nicer-looking, but I think they are also less flexible. In D there are many things that are less flexible than similar things in C++ (see for example operator overloading too), this can be a good thing, a design choice, because too much flexibility makes a design bug-prone, complex to implement and harder to use. So you often want less flexibility, even if in hard situations (like your one) such limits may look bad.

Bye and thank you for all your work so far,
bearophile