Something needs to happen with shared, and soon.

[Alex Rønne Petersen]

Hi,

It's starting to get outright embarrassing to talk to newcomers about 
D's concurrency support because the most fundamental part of it -- the 
shared type qualifier -- does not have well-defined semantics at all.

I'm certainly not alone in being annoyed by this state of affairs: 
http://d.puremagic.com/issues/show_bug.cgi?id=8993

I've posted rants about the state of shared before and, from the 
comments on those, it appears that what most people want shared to do is 
at least one (and usually multiple) of

* make variables global (if appropriate in the context);
* make the wrapped type completely separate from the unwrapped type;
* make all operations be atomic;
* make all operations result in memory barriers.

At a glance, this looks fine. Exactly what you would want for shared 
types in a concurrent setting, right?

Except, not really. I'll try to explain all of the unsolved problems 
with shared below...

First of all, the fact that shared(T) is completely separate from T 
(i.e. no conversions allowed, except for primitive types) is a huge 
usability problem. In practice, it means that 99% of the standard 
library is unusable with shared types. Hell, even most of the runtime 
doesn't work with shared types. I don't know how to best solve this 
particular problem; I'm just pointing it out because anyone who tries to 
do anything non-trivial with shared will invariably run into this.

Second, the idea of making shared insert atomic operations is an 
absolute fallacy. It only makes sense for primitive types for the most 
part, and even for those, what sizes are supported depends on the target 
architecture. A number of ideas have come up to solve this problem:

* We make shared(T) not compile for certain Ts depending on the target 
architecture. I personally think this is a terrible idea because most 
code using shared will not be portable at all.
* We require any architecture D targets to support atomic operations for 
a certain size S at the very least. This is fine for primitives up to 64 
bits in size, but doesn't clear up the situation for larger types (real, 
complex types, cent/ucent, ...).
* We make shared not insert atomic operations at all (thus making it 
kind of useless for anything but documentation).
* (Possibly others I have forgotten; please let me know if this is the 
case.)

I don't think any of these are particularly attractive, to be honest. If 
we do make shared insert atomic operations, we would also have to 
consider the memory ordering of those operations.

Third, we have memory barriers. I strongly suspect that this is a 
misnomer in most cases where people have suggested this; it's generally 
not useful to have a compiler insert barriers because they are used to 
control ordering of load/store operations which is something the 
programmer will want to do explicitly. In any case, the compiler can't 
usefully figure out where to put barriers, so it would just result in 
really bad performance for no apparent gain.

Fourth, there is implementation complexity. If shared is meant to insert 
specialized instructions, it will result in effectively two code paths 
for most code generation in any D compiler (read: maintenance nightmare).

Fifth, it is completely unclear whether casting to and from shared is 
legal (but with a big fat "caution" sign like casting away const) or if 
it's undefined behavior. Making it undefined behavior would further 
increase the usability problem I described above.

And finally, the worst part of all of this? People writing code that 
uses shared today are blindly assuming it actually does the right thing. 
It doesn't. Their code will break on any non-x86 platform. This is an 
absolutely horrifying situation now that ARM, MIPS, and PowerPC are 
starting to become viable targets for D.

Something needs to be done about shared. I don't know what, but the 
current situation is -- and I'm really not exaggerating here -- 
laughable. I think we either need to just make it perfectly clear that 
shared is for documentation purposes and nothing else, or, figure out an 
alternative system to shared, because I don't see shared actually being 
useful for real world work no matter what we do with it.

-- 
Alex Rønne Petersen
alex at lycus.org
http://lycus.org