Concurrency in D

[Robert Jacques]

On Tue, 24 Mar 2009 21:46:32 -0400, Jason House  
<jason.james.house at gmail.com> wrote:

> Bartosz Milewski Wrote:
>
>> Since I'm working on the concurrency model for D, I have to re-read  
>> several articles. I decided to combine this task with blogging--it  
>> always helps the understanding if you have to relate the topic to  
>> others. We are interested in designing a type system that would  
>> prevent, or at least minimize, data races in D. There's been research  
>> on this, especially in Java, so I started with analyzing Guava, a  
>> race-free dialect of Java. This is the post:
>> http://bartoszmilewski.wordpress.com/2009/03/23/types-for-concurrency/  
>> , and here's a link to reddit, if you want to vote for it:  
>> http://www.reddit.com/r/programming/comments/86zow/types_for_concurrency/  
>> .
>
> Any conclusions about what D should do?

I posted some comments to the blog, here’s an expanded version making the  
D connection:

Guava feels like it has over-complicated itself by using an object-centric  
abstraction.  Essentially, there are shared objects and objects local to  
those shared objects. (There’s no concept of D’s scope in Java as the VM  
dynamically makes this decision) Most of the discussion on D’s model is  
thread-centric: shared objects, thread-local objects and scope objects.

First, the good ideas
1. The concept of shared objects (Guava Monitors) being able to have and  
use private helper objects. While this is mainly about performance rather  
than correctness (at least in D), documenting and enforcing the concept of  
a member variable that must be contained within an object’s scope is  
useful, particularly for structs* since they have no protection of their  
own. Though not mentioned, the ability to debug with these promoted to a  
fully protected shared object, would help find bugs. The bugs should only  
come from poorly written functions inside the object’s module and analysis  
might be possible to prevent this (possibly also a compiler flag).  
Mentally, I’m calling these scope member variables since D already has the  
keyword and they cognitively share the same meaning: that this variable is  
restricted to the scope it’s declared in.
Rules:
1) Transitive
2) Assignable only by the ‘new’ statement: new’s behavior (i.e. where it  
allocates) is dependent on the internal scope type (scope_function,  
scope_local, scope_shared) which is determined by declaration location.
By the way, now that I think about it, scope member variables are similar  
in concept to mutable functions inside pure functions. And if you’re  
keeping count, that’s five ownership types, three of which are not  
programmer declarable and no ‘const’ equivalent in sight. (I posted about  
‘final’ variables a few days ago which attempts to solve this problem.)
2. kept/lent are neat, intuitive keyword names.
3. Although, I’ve seen it elsewhere as well, Guava uses a reader/writer  
monitor to protect access to shared objects. Hence, const (Guava read)  
members could be executed concurrently.

Next, the bad ideas
1. They didn’t break the scope member variables into a separate type. This  
results in a bunch of compiler analysis and some cognitive load on the  
programmer.
2. They have ‘new’ methods in which objects don’t have ownership as well  
as move semantics for value types. Think Unique!(T), since they allow  
objects as value types. Both of these mechanics types break the locality  
of memory allocation. This mean you can’t have local-garbage collection  
heaps which necessitates less efficient, less scalable and higher latency  
alternatives.
3. They use class centric region boundaries. This seems to add several  
layers of complexity without any benefit. Particularly, the kept/lent  
system exists solely because objects need to be passed across this  
boundary regularly and therefore doing so must be efficient. This seems  
like a hallmark that the wrong region boundaries where chosen. For  
instance, a thread-centric boundary avoids the need for a kept/lent system  
since the local objects don’t exit or enter the thread. (Thread-centric  
design also fixes bad idea #2, even Unique)