[dmd-concurrency] composability

Fri Jan 8 17:16:56 PST 2010

On Fri, Jan 8, 2010 at 7:53 PM, Kevin Bealer <kevinbealer at gmail.com> wrote:

> I'm thinking of the engineers here as people who want to build a system and
> aren't going to understand or safely know how to innovate with something
> like CAS and are just doing application code, and the scientists as people
> who are trying to build lock free algorithms and libraries and who do 'get'
> CAS.
>

To clarify further -- for someone who is 'just writing apps', often they
want to have two or three containers and protect them by the same mutex so
that the data among all containers is consistent with each other based on
some overall design-by-contract.

For example, suppose I have two hash tables with Customer records, one for
all customers and one for customers with open orders, and I want to delete a
Customer from both.  CAS can't protect both unless I knew this usage pattern
when I was writing the hash tables.  If I add a third table Orders with the
actual work orders, again it fails.

In this case, the customer would want to lock all the tables with a single
Mutex and let operations get the mutex, do all their stuff, and then release
it.  But now the CAS ops are doing extra work and throwing potentially many
more memory barriers than needed since the mutex already has it covered.

(I have some ideas on this but I'll send them later tonight.)

Maybe I'm wrong to be worried about this distinction -- maybe containers for
lock-free stuff are only used for special real-time stuff and are too
different than general purpose STL type containers.

(Given time constraints et al, if this issue seems like a side bar I can
drop it.)

> What I'm saying is that it would be great if these two groups didn't end up
> split into two completely different ways of doing business.  It would be
> good if the libraries that were created by the scientists were usable by
> people who were just doing application code.
>
> E.g. I can use my car without understanding the nanotech that triggers the
> air bags.  I who know little can benefit from the work of geniuses at Ford
> or Honda who do these incredible things, and they can benefit from having a
> large market for their product.
>
> In terms of D, it would be great if the concurrency was expressed in such a
> way that a person writing a billing application could use data structures
> implemented via the CAS and similar technologies, and the language had the
> tools to encapsulate or hide the hard mental-model problems at least for
> people who don't want to cope with them.
>
> The threat (if it can be called that) that I see is that we end up with two
> container libraries, one that is a highly technical piece of research that
> you need to understand lock-free and wait-free concepts to use, and the one
> that is a warmed over copy of the Java or STL libs (don't get me wrong, I
> like the STL most of the time) that the vast majority of consumers use but
> which doesn't and probably never will benefit from the CAS work.
>
> Kevin
>
>   On Fri, Jan 8, 2010 at 8:03 AM, Andrei Alexandrescu <andrei at erdani.com>wrote:
>
>> Nice essay, it just doesn't apply to this case; its converse does. There
>> has been a flurry of theoretical papers defining containers and algorithms
>> relying on DCAS and CASN. They died off because no one has been able to
>> implement those efficiently. The work that endured did it all with CAS.
>>
>> Andrei
>>
>> Kevin Bealer wrote:
>>
>>> On Thu, Jan 7, 2010 at 11:05 PM, Andrei Alexandrescu <andrei at erdani.com<mailto:
>>> andrei at erdani.com>> wrote:
>>>
>>>    Benjamin Shropshire wrote:
>>>
>>>        Andrei Alexandrescu wrote:
>>>
>>>            Benjamin Shropshire wrote:
>>>
>>>                OTOH that is just another form of the CAS 2 vs CAS 3
>>>                problem. The system only works if you have the right
>>>                abstractions.
>>>
>>>
>>>            What do you mean by CAS 2 and CAS 3?
>>>
>>>
>>>        The problem Kevin cited of their being 1 and 2 word CAS ops, but
>>>        no 3 word CAS, and if their was a 3 word CAS, then there not
>>>        being a 4 word CAS. My point being that you will end up always
>>>        with some case being just out of reach.
>>>
>>>
>>>    The good news is that CAS is all we need. True, it's not easy...
>>>
>>>
>>>    Andrei
>>>
>>>    _______________________________________________
>>>    dmd-concurrency mailing list
>>>    dmd-concurrency at puremagic.com <mailto:dmd-concurrency at puremagic.com>
>>>
>>>    http://lists.puremagic.com/mailman/listinfo/dmd-concurrency
>>>
>>>
>>> I think there are two camps here.  One is the academic / mathematical
>>> camp, that loves to tackle the "it's not easy but theory says it's
>>> possible."  The second camp is the "what are the best practices?" camp that
>>> doesn't own a microscope but looks at trade-offs between known designs.
>>>  Sort of the 'scientists' and the 'engineers'.
>>>
>>> The scientists usually fall into the role of studying fluid dynamics with
>>> an eye toward discovering the trick to making the new and experimental but
>>> elusive frictionless pipe.  The Engineers tend to study building codes and
>>> grunge through installing plumbing most of the time, with an eye toward
>>> becoming inventors or businessmen.  Most of the industry ends up playing the
>>> 'engineer' role.
>>>
>>> Personally I like tackling the "it's not easy" problems, or at least
>>> reading about them and pondering.  But what I'd ideally want to make sure is
>>> that both camps have a language they can work with.  That way, when the
>>> scientists discover the new breakthrough in fluid dynamics, it's more likely
>>> to be compatible with the toilets and dishwashers that everyone already has.
>>>
>>> (Okay, I clearly need sleep.)
>>>
>>> Kevin
>>>
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>>
>>> _______________________________________________
>>> dmd-concurrency mailing list
>>> dmd-concurrency at puremagic.com
>>> http://lists.puremagic.com/mailman/listinfo/dmd-concurrency
>>>
>>  _______________________________________________
>> dmd-concurrency mailing list
>> dmd-concurrency at puremagic.com
>> http://lists.puremagic.com/mailman/listinfo/dmd-concurrency
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/dmd-concurrency/attachments/20100108/a5dbf39b/attachment.htm>