Vision document for H1 2017

[Jonathan M Davis via Digitalmars-d-announce]

On Thursday, January 12, 2017 21:57:37 Andrew Browne via Digitalmars-d-
announce wrote:
> On Wednesday, 4 January 2017 at 19:22:33 UTC, Andrei Alexandrescu
>
> wrote:
> > We release a brief Vision document summarizing the main goals
> > we plan to pursue in the coming six months. This half we are
> > focusing on three things: safety, lifetime management, and
> > static introspection.
> >
> > https://wiki.dlang.org/Vision/2017H1
> >
> >
> > Andrei
>
> Is there a design document for how D will achieve safety with
> nogc?
> How does D plan to prevent leaks, use-after-free, double-free
> bugs when not using the GC?

Part of the reason that we have the GC in D is because of the safety
guarantees that you can have with a GC that you can't have with mechanisms
like malloc and free. Some amount of @nogc code can be @safe, but some of it
will never be able to be @safe. e.g. the very nature of malloc and free
makes @safe impossible in the general case. It's trivial for a piece of code
to free something that's currently in use by other code. If they're
constrained within a ref-counting system, then @safety becomes more
possible, but even then, when you have to start worrying about stuff like
weak references in order to get around circular reference problems, it gets
dicey if not impossible to make it fully @safe. It might be possible to
guarantee safety if you have a whole bunch of extra constraints like Rust
does with its borrowing stuff, but we're not going to add something like
that to D, because it's too complicated on top of everything else that we
already have.

I fully expect that certain idioms will be in place to help maintain @safety
in @nogc code, but to some extent, by abandoning the GC, you're abandoning
@safety - or at least you're making a lot more of your code need to be
@trusted, and you can rely less on the compiler to guarantee @safety for
you. Taking the freeing of memory out of the hands of the programmer like
happens with the GC is _huge_ in guaranteeing the memory safety of code.

> Will @nogc also have first class support in the language?

And what do you mean my first class support? Some features require the GC,
and I wouldn't expect it to ever be otherwise. Giving up the GC means giving
up on certain features. We don't want that list to be longer that it needs
to be, but some stuff fundamentally needs the GC to do what it does.

> Afaik the GC is currently needed for language features like array
> concatenation. Will features like array concatentation still work
> with @nogc?

I don't see how it possibly could given how dynamic arrays work in D. It
would have to have some sort of reference counting mechanism, which would
likely be a nightmare with slicing and certainly does not at all play well
with how low level D arrays are. We may very well get some sort of
ref-counted array type that has concatenation, but it would be a library
construct rather than in the language, because it doesn't need to be in the
language, and the built-in arrays would not be affected by it.

> GC allocations have a keyword 'new' (afaik 'new' currently never
> means anything other than GC allocation). Will we be able to do
> @nogc allocations by the 'new' keyword?

I very much doubt it. Constructing objects into memory is done via emplace,
which is a library construct, and there's really no need for it to be in the
language. As it is, if we were doing things from scratch, new probably
wouldn't even be a keyword. It would likely be a library construct in
druntime, because D is powerful enough that new doesn't need to be in the
language to do what it does. And in general, at this point, Walter and
Andrei don't want to put stuff in the language unless it actually needs to
be there. If it can be done with a library, it will be done with a library.
The only reason that they decided that we needed some sort of ref-counting
mechanism in the language is because they decided that it wasn't actually
possible to make it fully @safe without it being part of the language. And
even then, I'm not sure that the intention is that the ref-counting
mechanism use anything other than the GC. It's not yet clear what it's going
to look like, but previously, the talk was using the GC to take care of
circular references, which would mean that the memory was still GC-allocated
even if it were ref-counted. We'll have to wait and see though.

> Is the same code always expected to work with/without @nogc?

That would depend entirely on the code. std.experimental.allocator has a GC
allocator. So, code that is designed around it could work with the GC or
without. But the whole mechanism of newing something up and then not
worrying about ownership which happens by default with the GC doesn't play
at all nicely with how memory has to be managed via other mechanisms like
malloc and free. I don't think that it's at all reasonable to expect that
code that is written with the idea that its memory will be managed by the GC
will also work without the GC. Code that isn't managing memory directly
shouldn't care - and that's a lot of code (especially once lazy ranges are
in the mix) - but I think that it's pretty clear that there's plenty of code
that simply can't just swap out its allocation mechanism and still work
properly. In general, code needs to be written with that in mind for it to
work, and even then, there are limits given how different various memory
management mechanisms are.

- Jonathan M Davis