DMD 1.029 and 2.013 releases

[Walter Bright]

Sean Kelly wrote:
>>> Look at shared-
>>> memory multiprogramming for instance.  It's quite easy and understandable
>>> to share a few data structures between threads
>> It is until one of those threads tries to change the data.
> 
> I suppose I should have been more clear.  An underlying assumption of mine is
> that no thread maintains references into shared data unless they hold the lock
> that protects that data.

The problem with locks are:

1) they are expensive, so people try to optimize them away (grep for 
"double checked locking")
2) people forget to use the locks
3) deadlocks

>>> (which I'd assert is the original
>>> intent anyway), but common practice among non-experts is to use mutexes
>>> to protect code rather than data, and to call across threads willy-nilly.  It's
>>> no wonder the commonly held belief is that multiprogramming is hard.
>> The "multiprogramming is hard" is not based on a misunderstanding. It
>> really is hard.
> 
> My claim is that multiprogramming is hard because the ability to share memory
> has been mis-used.  It's not hard in general, in my opinion.

When people as smart and savvy as Scott Meyers find it confusing, it's 
confusing. (Scott Meyers wrote the definitive paper on doubled checked 
locking, and what's wrong with it.) Heck, I have a hard enough time 
explaining what the difference between const and invariant is, how is 
memory coherency going to go down? <g>


>>> Regarding lock-free programming in particular, I think it's worth pointing
>>> out that leaving out support for lock-free programming in general excludes
>>> an entire realm of code being written--not only library code to be ultimately
>>> used by everyday programmers, but kernel code and such as well.  Look at
>>> the Linux source code, for example.
>> I agree that lock free programming is important, but volatile doesn't
>> get you there.
> 
> How is it lacking?  I grant that it's very low-level, but it does address the key
> concern for lock-free programming.

volatile actually puts locks around accesses (at least in the Java 
memory model it does). So, it doesn't get you lock-free programming. 
Just avoiding caching of reloads is not the key to lock-free 
programming. There's the ordering problem.


>>  >  As for the C++0x discussions, I feel
>>  > that some of the participants of the memory model discussion are experts
>>  > in the field and understand quite well the issues involved.
>> Yes, there are a handful who do really understand it (Hans Boehm and
>> Herb Sutter come to mind). If only the rest of us were half as smart <g>.
> 
> My personal belief is that the issue is really more a lack of plain old explanation
> of the concepts than anything else.  The topic is rarely discussed outside of
> research papers, and most other documentation is either confusing or just plain
> wrong (the IA-86 memory model spec comes to mind, for example).  Not to
> belittle the knowledge or experience of the C++ folks in any respect--this is
> simply my experience with the information surrounding the topic :-)

I've seen many attempts at explaining it, including presentations by 
Herb Sutter himself. Sorry, but most of the audience doesn't get it.

I attended a conference a couple years back on what do do about adding 
multithreading support to C++. There were about 30 attendees, pretty 
much the top guys in C++ programming, including Herb Sutter and Hans 
Boehm. Herb and Hans did most of the talking, and the rest of us sat 
there wondering "what's a cubit". Things have improved a bit since then, 
but it's pretty clear that the bulk of programmers are never going to 
get it, and getting mp programs to work will have the status of a black art.

What's needed is something like what garbage collection did for memory 
management. The language has to take care of synchronization 
*automatically*. Being D, of course there will be a way for the 
sorcerers to practice the black art, but for the rest of us there needs 
to be a reliable and reasonable alternative.