DIP 1008 Preliminary Review Round 1

[Jonathan M Davis via Digitalmars-d]

On Friday, May 19, 2017 11:35:54 AM PDT H. S. Teoh via Digitalmars-d wrote:
> I agree with this, and having looked at std.experimental.allocator
> recently, I think Andrei may even have made certain design decisions
> with this in mind.  I think the ideal goal (I'm not sure how achievable
> it is) would be to make theAllocator part of *druntime*, upon which you
> can actually use array concatenations, AA's, etc., (almost) freely, and
> the user would be able to control exactly how allocations would work. At
> least in the current state of things, I don't see this as being
> particularly possible due to various issues, the main one being that
> code written with GC in mind generally does not track lifetimes, which
> is a big obstacle to successfully being able to just assign a different
> allocator to theAllocator and have things Just Work(tm).

Because of the issue of lifetimes, some language features simply cannot be
implemented without the GC, and I think don't see any point in trying to
make it so that you can use all features of D without the GC. That simply
won't work. By the very nature of the language, completely avoiding the GC
means completely avoiding some features.

D's dynamic arrays fundamentally require the GC, because they do not manage
their own memory. They're just a pointer and a length and literally do not
care what memory backs them. As long as all you're doing is slicing them and
passing them around (i.e. restrict yourself to the range-based functions),
then the GC is not involved, and doesn't need to be, but as soon as you
concatenate or append, the GC has to be involved. For that not to be the
case, dynamic arrays would have to manage their own memory (e.g. be
ref-counted), which means that they could not be what they are now. A
different data structure would be required.

Similarly, stuff like closures require the GC. They need something to manage
their memory. They're designed to be automatic.

Honestly, I think that this push for @nogc and manual memory mangement is
toxic. Yes, we should strive to not require the GC where reasonable, but
some things simply are going to require the GC to work well, and avoiding
the GC very quickly gives you a lot of the problems that you have with
languages like C and C++.

For instance, at dconf, Atila talked about the D wrapper for excel that he
wrote. He decided to use @nogc and std.exception.allocator, and not only did
that make it much harder for him to come up with a good, workable design, it
meant that he suddenly had to deal with memory corruption bugs that you
simply never have with the GC. He felt like he was stuck programming in C++
again - only worse, because he had issues with valgrind that made it so that
he couldn't effectively use it to locate his memory corruption problems.

The GC makes it far easier to write clean, memory-safe code. It is a _huge_
boon for us to have the GC. Yes, there are cases where you can't afford to
use the GC, or you have to limit its use in order for your code to be as
performant as it needs to be, but that's the exception, not the norm. And
avoiding the GC comes at a real cost.

I have no problem whatsoever telling folks that some features of D require
the GC and that while D's GC is optional, if you avoid it, you avoid certain
features. There is no free lunch.

We do need to make sure that we do not accidentally or erroneously require
the GC so that code can work with @nogc when folks require that if it's
reasonable for that code to be @nogc. Where reasonable, allocators should be
an option so that folks can decide whether to use the GC or not for a
particular piece of code. But sometimes, it's just not reasonable to expect
the same functionality when not using the GC as you get when using the GC.

And the reality of the matter is that using the GC has real benefits, and
trying to avoid it comes at a real cost, much as a number of C++ progammers
want to complain and deride as soon as they hear that D has a GC. And
honestly, even having @nogc all over the place won't make many of them
happy, because the GC is still in the language.

- Jonathan M Davis