[dmd-concurrency] D's Memory Model

[Robert Jacques]

On Wed, 10 Feb 2010 02:12:04 -0500, Andrei Alexandrescu  
<andrei at erdani.com> wrote:

> Robert Jacques wrote:
>> On Tue, 09 Feb 2010 22:23:05 -0500, Andrei Alexandrescu  
>> <andrei at erdani.com> wrote:
>>> Robert Jacques wrote:
>>> [snip]
>>>
>>> Just to make sure, I'm tuning out this entire GC-related discussion  
>>> because I assume it's not related to the core language. If there are  
>>> bits that prompt language changes please let me know. Thanks.
>>>
>>> Andrei
>>  Yes, there was one important language change which my first post  
>> advocated: the separation of local and shared (+immutable & unique)  
>> data into logically separate memory pools. I'm sorry that I didn't make  
>> it clearer. From a TDPL point of view, this allows implementations to  
>> eliminate false sharing (an important fact to highlight) and to use  
>> thread-local allocation and/or thread-local collectors. The downside is  
>> that it adds language complexity and makes casting from local to  
>> shared/immutable/unique implementation defined.
>
> Doing this would disallow people from transporting large amounts of data  
> across thread boundaries, which is now possible with cast and the  
> requisite care. I'm not sure we are ready to give that up.

The recommended way of transporting large amounts of data across thread  
boundaries is already to use immutable or unique data, both of which can  
and should be allocated from the shared memory pool. So while the vast  
majority of use cases would be covered, naturally there wile be exceptions.

>> Now, I've mentioned this enhancement to D before, in regard to  
>> thread-local garbage collectors, but Walter didn't think the  
>> cost/benefit was worth it at the time. Indeed, thread-local collectors  
>> are just one of many possible modern GCs a D runtime could adopt.  
>> Except, the follow-up discussion with Fawzi has started to give me a  
>> sinking feeling. All those wonderful modern GCs seem to be designed for  
>> functional or (mostly) pure-OO languages, not systems programming  
>> languages. So far, everything but thread-local GCs has gone down in  
>> flames. This is disconcerting as even TLGCs are meant to be paired with  
>> a modern shared GC of some kind. I'm not a GC expert, so there might be  
>> something out there, but I think it would be very prudent to find out  
>> what they are, in case they too require some change to the language to  
>> work.
>
> I'm not sure your concerns are valid. Most other OO languages are  
> messier than D with regard to concurrency, yet their garbage collectors  
> do manage.

Language concurrency and GC concurrency are very different beasts.

> I agree you have a good point, but the constraints within which we are  
> operating are forcing me to ignore this aspect unless a very clear  
> vision of the problem and a possible solution comes forward. (But by all  
> means feel free to continue the discussion.) From what I can tell after  
> spending little time (th|s)inking about this matter, it's certain casts  
> that thread-local mutable storage would disallow, and that is a  
> possibility that is easy to introduce later.

I understand and it is a minor change, as you pointed out. The only issue  
I saw was that a while ago it was recommended to build immutable  
objects/strings using local copies first and then casting. Although I  
think unique is now taking over this role.