shared adventures in the realm of thread-safety.

[Jeremie Pelletier]

Robert Jacques Wrote:

> On Sun, 13 Sep 2009 18:08:57 -0400, Jeremie Pelletier <jeremiep at gmail.com>  
> wrote:
> 
> > Robert Jacques Wrote:
> >
> >> On Sun, 13 Sep 2009 15:04:57 -0400, Jeremie Pelletier  
> >> <jeremiep at gmail.com>
> >> wrote:
> >> [snip]
> >> > Unique data could only be used for aggregate properties,  
> >> const/immutable
> >> > data would also be implicitly unique. This qualifier alone would
> >> > simplify shared quite a lot, allowing the use of unshared objects in
> >> > shared contexts safely.
> >>
> >> Neither const nor immutable data can be considered unique. First, any
> >> const data may be being mutated by another routine, so it can't be  
> >> safely
> >> accessed without synchronization. Second, unique data is mutable while
> >> const/immutable data is not. Third, most implementations of unique allow
> >> for deterministic memory reclamation, which isn't possible if the unique
> >> data might actually be const/immutable.
> >
> > Good points, I can only agree with you here. However I still believe  
> > immutable data should be able to be used in shared contexts without  
> > being 'shared' or protected by a monitor.
> 
> One of the purposes behind immutable was lock-free access. As far as I  
> know you can use immutable data in shared contexts today without any other  
> modifiers. A quick test seems to indicate this works today, but if you've  
> got a test case where it doesn't, I'd recommend filing it as a bug.

Oh yeah, I'm confusing it with 'shared' methods not able to call 'const shared' methods, which is a pain in the ass :(

> >> > The compiler should make the distinction between shared code and  
> >> shared
> >> > data and allow both shared and unshared instances to use shared  
> >> methods,
> >> > just like both const and mutable instances may call const methods. An
> >> > error should also be triggered when calling a shared method of a  
> >> shared
> >> > object without synchronization, and maybe have a __sync keyword to
> >> > override this. If a synchronized method is called from a non-shared
> >> > object, no synchronization takes place.
> >>
> >> I think you have the wrong paradigm in mind. Shared and non-shared  
> >> aren't
> >> mutable and const. They're mutable and immutable. From a technical
> >> perspective, synchronization of shared methods are handled by the  
> >> callee,
> >> so there is no way not to call them and non-shared objects don't have a
> >> monitor that can be synchronized. Now you can have the compiler use the
> >> same code to generate two different object types (vtables, object  
> >> layouts,
> >> etc) with have the same interface, but that doesn't sound like what  
> >> you're
> >> suggesting.
> >
> > I know that shared/unshared is not const/mutable. What I meant is that  
> > right now in D if a method is 'shared' it cannot be called from a  
> > non-shared object, which makes unshared instance of the class unusable  
> > without plenty of dirty casts. Take the following objects:
> >
> > class Foo { void foo() const; }
> > class Bar { void bar() shared; }
> >
> > Foo foo; foo.foo(); // ok, mutable object can call const method
> > Bar bar; bar.bar(); // error, unshared object may not call shared method
> >
> > I had only presented the concept, your idea of using two virtual tables  
> > for shared/unshared instances is also what I had in mind for the  
> > implementation, and it would give exactly the behavior I had in mind.
> 
> Bartosz took the concept one step further: when declared as shared, all  
> methods are implicitly wrapped in synchronize blocks. He then added a  
> keyword for more manual, lock-free style programming. But this syntactic  
> sugar isn't implemented yet.

The current D keywords (synchronized and shared) are already designed for that, since synchronized implies shared. I don't want implicit synchronization, I'd much rather have a shared class marking all its members/properties as shared and letting me explicitely decide where the synchronization takes place.

> >> > Allow me to illustrate my point with some code:
> >> >
> >> > class Foo {
> >> >     int bar() shared { return a; }
> >> >     __sync bar2() { synchronized(this) return a; }
> >> >     synchronized void foo() { a = 1; }
> >> >     int a;
> >> > }
> >> > auto foo1 = new shared(Foo)();
> >> > auto foo2 = new Foo;
> >> >
> >> > foo1.foo(); // ok, synchronized call
> >> > synchronized(foo1) foo1.foo(); // warning: recursive synchronization
> >>
> >> Why a warning? Monitors are designed to handle recursive  
> >> synchronization.
> >
> > Its a performance issue that can easily be avoided, but still generates  
> > valid code.
> 
> Really? Every public method that calls another public method (of the same  
> object) results in recursive synchronization. And if your example was  
> longer than a one liner, you'd also have to have recursive  
> synchronization. There are ways to reduce recursive synchronization, like  
> public wrappers of protected/private methods, but they are not always  
> appropriate or feasible for the use case. BTW, in general the threshold  
> for what's a warning in DMD is generally a lot higher than other compilers  
> (on the theory that if warnings are generated for every build you'll never  
> read them)

Its a behavior that is really easy to avoid, and therefore the overhead easily avoided. The custom runtime I use doesn't use the reentrant attribute on pthread's mutexes and recursing into a monitor triggers a runtime exception, this is by design to better optimize the code. On Windows critical sections are sadly always reentrant.

I haven't come across any case I wasn't able to easily design to avoid recursive mutexes yet.

> [snip]
> 
> >> Bike-shed: I've always preferred the CSP/pi-calculas term 'mobile' for  
> >> the
> >> concept of 'unique'. I think mobile better expresses the concept with
> >> regard to multi-threading, where mobile is used to cheaply transfer data
> >> between threads (i.e. it moves around/can move between threads, but  
> >> isn't
> >> shared between them). I find 'unique' to mainly convey the memory  
> >> storage
> >> aspect of the concept, which is less important outside of C/C++.
> >
> > Maybe this is where 'volatile' could come back, from what I know it's  
> > still a reserved keyword in D and would fit nicely this purpose.
> 
> The volatile keyword has a very precise meaning in C/C++, which D altered  
> and then abandoned. I think using it for the concept of mobile/unique  
> would confusing. It also lacks any connotations related to a mobile/unique  
> type. (i.e. I don't see the logic behind the choice, besides the keyword  
> being unused)

Ok, maybe not the greatest idea of all times, I agree :)