[dmd-concurrency] synchronized, shared, and regular methods inside the same class

[Sean Kelly]

On Jan 4, 2010, at 9:00 AM, Andrei Alexandrescu wrote:

> Sean Kelly wrote:
>> I'll admit that I'm leery of having shared methods in general.
>> Operations occurring in order can still easily produce unexpected
>> results when multiple threads are interleaving execution.  Consider
>> increment, for example: if "x++" is executed as "x = x + 1" (ie. a
>> load, add, and a store).  I know that lock-free variables are
>> somewhat of a separate topic, but I wanted to mention this now
>> anyway.
> 
> Shared methods are necessary for enabling lock-free structures. For example a lock-free singly-linked list (a very useful structure) would have all of its methods shared.

My mistake.  I forgot that the shared label on methods is for visibility, since unlabeled methods won't be accessible from a shared reference.

> Read-modify-write operations a la x++ do not work for shared objects. I was surprised to find out just now that dmd does not enforce that so I just added http://d.puremagic.com/issues/show_bug.cgi?id=3672.
> 
> The only things that work on shared objects are:
> 
> * calls to synchronized or shared methods, if any;
> 
> * reading if the object is word-size or less;
> 
> * writing if the object is word-size or less.

Cool!  It's perhaps a minor issue right now, but it would be nice if RMW operations could be performed via library functions.  Hopefully all that's required is to accept a "ref shared T" and then write ASM for the machinery from there?  ie. Is there any need for compiler changes to support this?

>>> I think the compiler should enforce that a class either defines
>>> synchronized methods, shared methods, but not both. So the class
>>> writer must decide upfront whether they use lock-based or lock-free
>>> threading with a given class. Does this make sense? Is it too
>>> restrictive?
>> See above.  This would certainly be the easiest for the compiler to
>> verify though, since it wouldn't have to reason about the interaction
>> between lock-free operations and synchronized operations.  But
>> perhaps this is time for a rather large set of questions: can someone
>> define a shared non-class variable?
> 
> Absolutely. A global shared int or a global shared pointer are typical examples.
> 
>> If so, how would such a variable
>> work within shared and synchronized methods in these classes?
> 
> It would obey the regular shared rules. In case you're thinking about a field, shared or synchronized methods don't change the type of an object, subject to the "tail-shared" exemption that I'll discuss at a later point.

So if I have:

    class A
    {
        void fn() shared { x = 5; }
        int x;
    }

Is this legal?  If the type of the object doesn't change then I'd guess that I won't be allowed to access non-shared fields inside a shared function?

>> Would
>> shared methods only allow operations on shared variables?  Or perhaps
>> make all operations on data inside them shared regardless of how
>> they're labeled?
> 
> If a method is shared, it must assume "this" is shared. The fact that "this" was not shared upon invoking the method is an information that's lost.
> 
> By the way - you should be able to call shared methods against a non-shared object.
> 
>>> Walter asked - what if we take this one step further and force a
>>> class to define _all_ methods either synchronized, shared, or
>>> neither? In that case the keyword could be moved to the top:
>>> shared class A { ... all methods shared ... } synchronized class B
>>> { ... all methods synchronized ... } class C { ...
>>> shared/synchronized not allowed ... }
>>> Has experience with Java shown that it's best to design classes for
>>> concurrency in separation from single-threaded classes?
>> Tricky question.  In general I'd say yes, but that it stinks to have
>> two classes for the same thing that only differ by synchronization.
> 
> In the meantime, according to what I've read, that's exactly what Java containers do. Java containers stink already, are you adding some intensity to the stench?

I really don't know Java containers very well so I can't accurately quantify the stench, but it seems like a horrible duplication of code to do this, particularly in a language that doesn't have the facilities to wrap an arbitrary class automatically.

>> Consider a container class for example.  Standard operations (insert,
>> remove, etc) can have a "synchronized" label slapped on them and work
>> perfectly, but iterators are a different story entirely.
> 
> This supports Walter's idea that at best a class is designed from the get-go to be synchronized or not. Is that correct?

Yes I think so.

>>> Effective Java doesn't mention that. But then there are all these
>>> methods synchronizedCollection, synchronizedList etc. that wrap the
>>> corresponding collections (which probably don't have synchronized
>>> methods). I'm reading this up, but if there's someone on this list
>>> who could save us all some time, please chime in.
>> With D's metaprogramming capabilities, this is a reasonable option in
>> some cases.  Though see my comment about iterators above (foreach is
>> obviously fine though). 
> 
> You mean foreach defined as synchronized opApply?

Yes exactly.