[dmd-concurrency] Vot de hekk is shared good for, anyway?

[Graham St Jack]

Sean Kelly wrote:
> The send and receive routines for message passing will just be 
> template functions, so their implementation is totally hidden from the 
> user.  It shouldn't matter in the least ifthe underlying 
> implementation uses shared or not.  They may be able to optimize 
> passing shared data better by avoiding a copy but this would just 
> happen.  My hope is that the typical application will use shared in 
> it's own code little to not at all.
I'm happy with things being carefully wrapped and auto-magic. However, 
surely the message channel (or whatever it is called) object itself will 
have be shared (and its externally accessible methods synchronized), 
otherwise we are cheating. We have a object whose sole purpose in life 
is to be shared by multiple threads, so surely if anything is shared, 
this one has to be.

If this is the case, then we need a mechanism to stop "shared" from 
leaking out into the whole code base.

Since no-one ever responds to this line of questioning, it looks like I 
am missing something fundamental. Can anyone help me out here?

>
> Sent from my iPhone
>
> On Jan 10, 2010, at 3:16 PM, Graham St Jack 
> <graham.stjack at internode.on.net> wrote:
>
>> This all sounds excellent to me.
>>
>> A question:
>> How will the message-passing API avoid exposing "shared"? Surely the 
>> message channel itself is a shared mutable object, even though the 
>> messages passing through it are immutable. If it isn't shared, aren't 
>> we cheating by hiding the fact that multiple threads are accessing it?
>>
>>
>> Walter Bright wrote:
>>>
>>> There's another aspect here. Consider all the problems we have 
>>> getting across the idea of an immutable type. What hope is there for 
>>> shared? I see mass confusion everywhere. Frankly, I see little hope 
>>> of any but a handful of programmers ever being able to grok shared 
>>> and use it correctly for concurrent programs. The notion that one 
>>> can just slap 'shared' on a data type and have it work correctly 
>>> across threads without further thought is a pipe dream.
>>>
>>> So what to do?
>>>
>>> I want to pin the mainstream concurrency on message passing. The 
>>> message passing user never sees shared, never has to deal with 
>>> locks, never has to deal with memory barriers. It just works. 
>>> Message passing should be a robust, scalable solution for most 
>>> users. I believe the Erlang experience validates this. Go and Scala 
>>> also rely entirely on message passing (but they don't have immutable 
>>> data, so their models are unsafe and I predict many rude surprises).
>>>
>>> So why bother with shared at all?
>>>
>>> Because message passing does not cover all the bases, and D is 
>>> supposed to be a systems programming language. So we need a paradigm 
>>> for synchronization and shared data structures. What shared provides 
>>> is:
>>>
>>> 1. A way to identify shared data. This is incredibly important. A 
>>> lot of sharing bugs come about because of inadvertant unrecognized 
>>> sharing of data. This should be pretty much impossible in D. 
>>> Furthermore, if you do have a sharing bug in your code, you look at 
>>> the 1% of the data tagged as shared, rather than every freakin' line 
>>> of code and every piece of data. Half the battle in debugging code 
>>> is figuring out where to look for the problem. Shared pares that 
>>> problem down to a reasonable size.
>>>
>>> 2. Shared comes with a collection of static typing rules and 
>>> guarantees that will head off a number of concurrency bugs, such as 
>>> sequential consistency.
>>>
>>> I view shared as sort of like the latest electric arc welders which 
>>> automatically adjust the current and wire feed for you. They 
>>> dramatically shorten (but don't eliminate) the learning curve for 
>>> people trying to master the art of welding. D is the only language 
>>> to even attempt this. C++ leaves you completely on your own, Java 
>>> offers no help, Erlang, Scala and Go throw in the towel and won't 
>>> allow anything but message passing.
>>>
>>> As for a shared gc vs thread local gc, I just see an awful lot of 
>>> strange irreproducible bugs when someone passes data from one to the 
>>> other. I doubt it's worth it, unless it can be done with compiler 
>>> guarantees, which seem doubtful.
>>> _______________________________________________
>>> dmd-concurrency mailing list
>>> dmd-concurrency at puremagic.com
>>> http://lists.puremagic.com/mailman/listinfo/dmd-concurrency
>>
>> _______________________________________________
>> dmd-concurrency mailing list
>> dmd-concurrency at puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/dmd-concurrency
> _______________________________________________
> dmd-concurrency mailing list
> dmd-concurrency at puremagic.com
> http://lists.puremagic.com/mailman/listinfo/dmd-concurrency