Something needs to happen with shared, and soon.

[Michel Fortin]

On 2012-11-15 16:08:35 +0000, Dmitry Olshansky <dmitry.olsh at gmail.com> said:

> 11/15/2012 8:33 AM, Michel Fortin пишет:
> 
>> If you want to declare the mutex separately, you could do it by
>> specifying a variable instead of a type in the variable declaration:
>> 
>>      Mutex m;
>>      synchronized(m) int i;
>> 
>>      synchronized(i)
>>      {
>>          // implicit: m.lock();
>>          // implicit: scope (exit) m.unlock();
>>          i++;
>>      }
> 
> While the rest of proposal was more or less fine. I don't get why we 
> need escape control of mutex at all - in any case it just opens a 
> possibility to shout yourself in the foot.

In case you want to protect two variables (or more) with the same 
mutex. For instance:

	Mutex m;
	synchronized(m) int next_id;
	synchronized(m) Object[int] objects_by_id;

	int addObject(Object o)
	{
		synchronized(next_id, objects_by_id)
			return objects_by_id[next_id++] = o;
	}

Here it doesn't make sense and is less efficient to have two mutexes, 
since every time you need to lock on next_id you'll also want to lock 
on objects_by_id.

I'm not sure how you could shoot yourself in the foot with this. You 
might get worse performance if you reuse the same mutex for too many 
things, just like you might get better performance if you use it wisely.


> But anyway we can make it in the library right about now.
> 
> synchronized T ---> Synchronized!T
> synchronized(i){ ... } --->
> 
> i.access((x){
> //will lock & cast away shared T inside of it
> 	...
> });
> 
> I fail to see what it doesn't solve (aside of syntactic sugar).

It solves the problem too. But it's significantly more inconvenient to 
use. Here's my example above redone using Syncrhonized!T:

	Synchronized!(Tuple!(int, Object[int])) objects_by_id;

	int addObject(Object o)
	{
		int id;
		objects_by_id.access((obj_by_id){
			id = obj_by_id[1][obj_by_id[0]++] = o;
		};
		return id;
	}

I'm not sure if I have to explain why I prefer the first one or not, to 
me it's pretty obvious.


> The key point is that Synchronized!T is otherwise an opaque type.
> We could pack a few other simple primitives like 'load', 'store' etc. 
> All of them will go through lock-unlock.

Our proposals are pretty much identical. Your works by wrapping a 
variable in a struct template, mine is done with a policy object/struct 
associated with a variable. They'll produce the same code and impose 
the same restrictions.


> Even escaping a reference can be solved by passing inside of 'access'
> a proxy of T. It could even asserts that the lock is in indeed locked.

Only if you can make a proxy object that cannot leak a reference. It's 
already not obvious how to not leak the top-level reference, but we 
must also consider the case where you're protecting a data structure 
with the mutex and get a pointer to one of its part, like if you slice 
a container.

This is a hard problem. The language doesn't have a solution to that 
yet. However, having the link between the access policy and the 
variable known by the compiler makes it easier patch the hole later.

What bothers me currently is that because we want to patch all the 
holes while not having all the necessary tools in the language to avoid 
escaping references, we just make using mutexes and things alike 
impossible without casts at every corner, which makes things even more 
bug prone than being able to escape references in the first place.

There are many perils in concurrency, and the compiler cannot protect 
you from them all. It is of the uttermost importance that code dealing 
with mutexes be both readable and clear about what it is doing. Casts 
in this context are an obfuscator.


> Same goes about Atomic!T. Though the set of primitives is quite limited 
> depending on T.
> (I thought that built-in shared(T) is already atomic though so no need 
> to reinvent this wheel)
> 
> It's time we finally agree that 'shared' qualifier is an assembly 
> language of multi-threading based on sharing. It just needs some safe 
> patterns in the library.
> 
> That and clarifying explicitly what guarantees (aside from being well.. 
> being shared) it provides w.r.t. memory model.
> 
> Until reaching this thread I was under impression that shared means:
> - globally visible
> - atomic operations for stuff that fits in one word
> - sequentially consistent guarantee
> - any other forms of access are disallowed except via casts

Built-in shared(T) atomicity (sequential consistency) is a subject of 
debate in this thread. It is not clear to me what will be the 
conclusion, but the way I see things atomicity is just one of the many 
policies you may want to use for keeping consistency when sharing data 
between threads.

I'm not trilled by the idea of making everything atomic by default. 
That'll lure users to the bug-prone expert-only path while relegating 
the more generally applicable protection systems (mutexes) as a 
second-class citizen. I think it's better that you just can't do 
anything with shared, or that shared simply disappear, and that those 
variables that must be shared be accessible only through some kind of 
access policy. Atomic access should be one of those access policies, on 
an equal footing with other ones.

But if D2 is still "frozen" -- as it was meant to be when TDPL got out 
-- and only minor changes can be made to it now, I don't see much hope 
for its concurrency model. Your Syncronized!T and Atomic!T wrappers 
might be the best thing we can hope for, but they're nothing to set D 
apart from its rivals (I could implement that easily in C++ for 
instance).

-- 
Michel Fortin
michel.fortin at michelf.ca
http://michelf.ca/