What's the go with the GC these days?

[Jonathan M Davis]

On Saturday, January 5, 2019 3:05:19 PM MST Manu via Digitalmars-d wrote:
> Is progress possible, or is the hard reality that the language is just
> designed such to be resistant to a quality GC, while the ecosystem
> sadly tends to rely on it?

There are some fundamental limitations put on a GC for D, because D is a
systems language and is not particularly limited in what it can do,
rendering a lot of the stuff that languages like Java or Go do with their GC
impossible. And it's not that uncommon in GC discussions that someone
suggests something that won't work even though it initially seems like it
should - e.g. having separate memory pools for each thread doesn't work with
how D currently works; at minimum, how casting to and from shared and
immutable works would have to be changed (which would impact performance).
And at one point, Daniel Murphy explained some other technical hurdles that
he didn't seem to think were surmountable, but unfortunately, I can't
remember the details now - something to do with the vtable and TypeInfo, I
think, but I could be misremembering. In any case, a lot of the stuff that
would typically be done to improve a GC and make it more modern is off the
table with D.

But that doesn't mean that D's GC can't be improved. Work has been done on
that - e.g. Martin Nowak was going to do a talk at dconf 2015 about the
improvements to the GC that he had done, but he missed his plane, so we
unfortunately didn't get to hear that talk. Other work has been done towards
GC improvements, some of which has been merged, and some of which hasn't.
For instance, IIRC, Rainer made improvements to how precise the GC was a
year or two back (though I don't think that it's yet fully precise, and I
don't know how possible it is make it fully precise). Also, destructors now
get called for structs on the heap in many cases, whereas before, that
pretty much never happened. Other work has been done towards more radical
changes to the GC, though I don't think that any of that has made it in
(e.g. the forking GC that Sociomantic worked on which significantly reduced
how long the world got stopped - but AFAIK, no one has yet figured out how
to make that work on Windows, so none of that work has made it into
druntime). I don't think that a lot of work has been done towards improving
the GC (at least that has made it in yet), but improvements have been made.
They just don't get advertised much when they do, and it's a favorite
assumption that GC performance sucks. We can and should improve the GC, but
it requires that folks spend time on doing so, and it's not easy.

And really, for the most part, it's just folks saying that the GC is a
problem without any real evidence to back it up, whereas there are a number
of cases of folks in this newsgroup talking about how they used the GC for
stuff that was highly performance sensitive, and all they had to do was
avoid having it be used in some particular piece of code in order to make it
so that their programs were quite performant. I don't think that there's any
question that some types of workloads are heavily impacted by the GC, but if
anything, experience seems to show that D's GC isn't a performance problem
in most cases (particularly with how typical it is for D code to use structs
on the stack where possible rather than allocating on the heap), and when it
is a problem, it's often possible to adjust sections of your code so that
the GC isn't a problem anymore. But for the most part, what we have to go on
is just anecdotal evidence. There isn't much in the way of actual
performance comparisons with the GC (at least not that get made public), and
it's just a common assumption that D's GC is bad and hurts performance. How
true that really is in practice is an open question. But I think that how
true that is is going to depend heavily on what your code is doing, and D's
idioms do tend to lean towards minimizing heap usage of any kind, which
usually improves performance regardless of whether you're talking about
avoiding allocations from the GC heap or from somewhere else.

> Where's the ARC stuff? What happened to opAddRef/opRelease?

I don't know that we're ever going to get ARC specifically, but as I
understand it, whatever reference counting stuff Walter was working on got
put on hold for DIP 1000, because he needed that in order to make the
reference counting stuff @safe. There's also the issue of copy constructors
and Andrei's __mutable proposal which could impact what happens (since right
now, as soon as const or immutable get involved reference counting doesn't
work). So, basically, language improvements needed to be made in order to
make any kind of built-in reference-counting work (at least with @safe or
with const/immutable), and I don't know how much more needs to be completed
before a reference-counting mechanism can be added. At minimum, DIP 1000
will have to be completed.

- Jonathan M Davis