Very limited shared promotion

Jonathan M Davis newsgroup.d at jmdavisprog.com
Wed Jun 19 10:50:45 UTC 2019


On Wednesday, June 19, 2019 3:56:18 AM MDT Ola Fosheim Grøstad via 
Digitalmars-d wrote:
> On Wednesday, 19 June 2019 at 08:45:10 UTC, Jonathan M Davis
>
> wrote:
> > scope doesn't care where the object is stored. If something is
> > scope, then you can't take any references to it in @safe code.
> > The whole point is to ensure that no references escape. Whether
> > the objects are GC-allocated are not is irrelevant.
>
> So, the same as this:
> https://dlang.org/spec/function.html#return-scope-parameters
>
> Does this mean then that when you obtain write access to an
> object that "write access reference" will be scope qualified?
>
> And then you cannot use that reference with any function that
> does not have scope qualified parameters?

I don't follow. What do you mean by obtain write access? If you're talking
about shared, scope doesn't care. It just makes sure that references can't
escape in @safe code. Currently, scope does nothing special with shared at
all. All scope does is prevent escaping in @safe code. You use the variables
pretty much the same as normal otherwise, though the fact that you can't
escape any references can potentially get very restrictive.

> So basically all library code has to add "scope" to all its
> parameters whether it is written with shared in mind or not.
>
> So, "scope" becomes like "pure". Something that ought to be the
> default, but has to be added manually to all function prototypes
> in order to make safe multi-threaded programming less annoying.

Actually, using scope when you don't need to would be a serious problem. For
instance, scope does _not_ interact well with range-based code. The typical
thing for a lot of range-based functions to do is to take the argument, wrap
it in a struct that's a new range, and return it, whereas if the parameter
were scope, then you couldn't put the argument or any reference to it in the
struct.

> Or will the compiler automatically deduce that a function
> parameter is fulfilling the "scope" requirements even when it has
> not been specified?
>
> I guess it should, otherwise you'll end up with the "const"
> transition in C++, where you had to do ugly cast-hacks when
> calling functions that did treat parameters as const but the
> function signature had not specified it.

AFAIK, the only time that scope is inferred is when auto is used for a
variable declaration, and the value assigned to it is already scope. And
being unable to escape references is a big enough hinderance in many cases
that I don't know how useful it's really going to be in practice outside of
code that simply operates on pointers. How useful it's going to be in
practice is really an open question at this point.

> > implicitly cast to or from shared. As it is, in general, D
> > can't even know if there are multiple references to the same
> > object. Without that kind of information, it's not even
> > possible to know whether it's safe to pass an object from one
> > thread to another.
>
> You can do this with dataflow in many cases, maybe in most useful
> cases. Although something like unique_ptr in C++ helps (by
> convention).

Walter is against data flow analysis in most cases largely because when
language semantics depend on it, you have to specify exactly how the data
flow analysis works, and it's never clear exactly how far you should go with
it. He routinely shoots down features that would require data flow analysis.

> > be necessary to make that happen, because it would require
> > significant changes to how D works.
>
> Maybe, although I wonder how much you could do as library code.
> So I don't know.
>
> If metaprogramming is going to be the main focus of D then it
> makes sense to focus on allowing library authors to inform the
> type system of what their library types are capable of. Basically
> the opposite of a traditional type system where the typesystem
> checks constraints.
>
> So, the basic idea would be that the library tells the type
> system "my code provide these guarantees", rather than the the
> library asking the type system to verify that a set of guarantees
> hold.

I have no clue how feasible that would be. I'm also not sure what benefit
there would be in telling the compiler that a piece of code guarantees
something that the compiler doesn't already check for. The closest to that
that we currently have is @trusted, which allows code that the compiler
couldn't guarantee was @safe to be treated as @safe on the basis that the
programmer vetted it. But if @safe wasn't something that the compiler
checked already, then indicating to the compiler that a piece of code was
vetted by the programmer for memory safety doesn't seem very useful.

It has been proposed in the past that folks wanted to use UDAs to indicate
stuff that was somehow then checked by the compiler, but I don't know
exactly what the expectation there was. Certainly, there are bound to be
things that we could do along those lines, but a concrete proposal would be
necessary.

> > casting. _Maybe_, there will be a few, small places like with
> > what Manu is suggesting here where we will be able to leverage
> > what the compiler knows to implicitly convert something, but in
> > general, that's really not going to work.
>
> I think that make such special cases is a bit dangerous. It is
> really going down the path of C++ that leads to a situation where
> you only appeal to the same audience as C++.
>
> It also leaves the field open to Rust. I agree that borrowing
> might be tedious for memory safety, but might be generally
> suitable for making safer multi-threading available to more
> programmers. At least on the surface level, but I haven't spent a
> lot of time thinking about the limitations of borrow-checking in
> relation to multi-threading. So, it is just a hunch.
>
> I think it would be a mistake to outright dismiss tracking
> references.  There might be libraries available that can do it
> for you even, e.g. provide a compiler switch that enables the
> compiler to generate a graph that is passed onto an external
> library that does the checking.

Honestly, given how D's type system works, I don't think that it's really
possible to do anything substantially different from C++ as far the built-in
concurrency stuff goes. That being said, that doesn't mean that libraries
can't be built on top of that which make things nicer. How far we can go
with that, I don't know, but I think that it's pretty clear at this point
that shared itself is going to be pretty low level in terms of what it does.
It's mostly about segregating the data that's shared across threads and
preventing operations that are clearly not thread-safe - and that in of
itself is already a considerable improvement over C++. Regardless, before we
can be sure of nicer features that we might be able to build on top of
shared, we first need to get the low level aspects sorted out.

> > Even getting something like DIP 1000 has been a major ordeal,
> > and I don't think that it would have ever happened unless
> > Walter had been convinced that it was absolutely needed for
> > @safe code to be able to do stuff like reference counting.
>
> DIP1000 looks too limiting, but I don't know what it has enabled
> and where it falls short in practice.
>
> I guess strength and weaknesses will show up as people build new
> frameworks around it.

Yeah. We'll have to see. It's clearly of some use with pointers in
particular, but from what I've seen thus far, I suspect that it's simply too
restrictive to be very useful in general. We'll just have to wait and see
though.

- Jonathan M Davis






More information about the Digitalmars-d mailing list