More radical ideas about gc and reference counting

[Manu via Digitalmars-d]

On 3 May 2014 18:49, Benjamin Thaut via Digitalmars-d
<digitalmars-d at puremagic.com> wrote:
> Am 30.04.2014 22:21, schrieb Andrei Alexandrescu:
>>
>> Walter and I have had a long chat in which we figured our current
>> offering of abstractions could be improved. Here are some thoughts.
>> There's a lot of work ahead of us on that and I wanted to make sure
>> we're getting full community buy-in and backup.
>>
>> First off, we're considering eliminating destructor calls from within
>> the GC entirely. It makes for a faster and better GC, but the real
>> reason here is that destructors are philosophically bankrupt in a GC
>> environment. I think there's no need to argue that in this community.
>>
>> The GC never guarantees calling destructors even today, so this decision
>> would be just a point in the definition space (albeit an extreme one).
>>
>> That means classes that need cleanup (either directly or by having
>> fields that are structs with destructors) would need to garner that by
>> other means, such as reference counting or manual. We're considering
>> deprecating ~this() for classes in the future.
>>
>> Also, we're considering a revamp of built-in slices, as follows. Slices
>> of types without destructors stay as they are.
>>
>> Slices T[] of structs with destructors shall be silently lowered into
>> RCSlice!T, defined inside object.d. That type would occupy THREE words,
>> one of which being a pointer to a reference count. That type would
>> redefine all slice primitives to update the reference count accordingly.
>>
>> RCSlice!T will not convert implicitly to void[]. Explicit cast(void[])
>> will be allowed, and will ignore the reference count (so if a void[]
>> extracted from a T[] via a cast outlives all slices, dangling pointers
>> will ensue).
>>
>> I foresee any number of theoretical and practical issues with this
>> approach. Let's discuss some of them here.
>>
>>
>> Thanks,
>>
>> Andrei
>
>
> Honestly, that sounds like the entierly wrong apporach to me. Your
> approaching the problem in this way:
>
> "We can not implement a propper GC in D because the language design prevents
> us from doing so. So lets remove destructors to migate the issue of false
> pointers."
>
> While the approach should be.
>
> "The language does not allow to implement a propper GC (anything else then
> dirty mark & sweep), what needs to be changed to allow a implementation of a
> more sophisticated GC."

Couldn't agree more.
Abandoning destructors is a disaster.
Without destructors, you effectively have manual memory management, or
rather, manual 'resource' management, which is basically the same
thing, even if you have a GC.
It totally undermines the point of memory management as a foundational
element of the language if most things are to require manual
release/finalisation/destruction or whatever you wanna call it.


> Also let me tell you that at work we have a large C# codebase which heavily
> relies on resource management. So basically every class in there inherits
> from C#'s IDisposable interface which is used to manually call the finalizer
> on the class (but the C# GC will also call that finalizer!). Basically the
> entire codebase feels like manual memory management. You have to think about
> manually destroying every class and the entire advantage of having a GC,
> e.g. not having to think about memory management and thus beeing more
> productive, vanished. It really feels like writing C++ with C# syntax. Do we
> really want that for D?

This is interesting to hear someone else say this. I have always found
C# - an alleged GC language - to result in extensive manual memory
management in practise too.
I've ranted enough about it already, but I have come to the firm
conclusion that the entire premise of a mark&sweep GC is practically
corrupt. Especially in D.
Given this example that you raise with C#, and my own experience that
absolutely parallels your example, I realise that GC's failure extends
into far more cases than just the ones I'm usually representing.

I also maintain that GC isn't future-proof in essence. Computers grow
exponentially, and GC performance inversely tracks the volume of
memory in the system. Anything with an exponential growth curve is
fundamentally not future-proof.
I predict a 2025 Wikipedia entry: "GC was a cute idea that existed for
a few years in the early 2000's while memory ranged in the 100's mb -
few gb's, but quickly became unsustainable as computer technology
advanced".


> And what if I want unsafe slices of structs with destructors, for
> performance? Maybe I perfectly know that the memory behind the slice will
> outlive the slice, and I don't want the overhead of all the reference
> counthing behind it?
>
> If you actually deprecate ~this, there would be two options for me.
> 1) Migrate my entire codebase to some user defiend finalizer function (which
> doesn't have compiler support), which would be a lot of work.

Does ~this() actually work, or just usually work?
Do you call your destructors manually like C#?

> 2) Quit D. (which is becomeing more and more an option when reading the
> recent news group discussions.)

I'm starting to fear the same outcome for myself.
I don't have any idea how to reconcile this problem in my working
environment, and very little community sympathy. I'm not seeing real
solutions emerging, and the only one I can imagine that's even
theoretically possible is wildly unpopular (ARC).
For years, I just (naively?) assumed that the GC was immature, and
would improve with time. Never gave it much thought; assumed there
were people much smarter than me with a plan...

I can't imagine any way out of this without significant breaking
changes to the type system. Probably a new pointer type at least.


This thread is starting to convince me that the GC should probably be
repositioned as a convenience library provided *beside* the language,
rather than a foundation of the language. It should be exclusively
opt-in, not opt-out, and upon opting-in, you accept the associated
problems.
The revelation that established GC languages like C# are actually
broken too hadn't occurred to me until now, but if I had to nominate
the single biggest disaster in C# from my experience, that's certainly
it; it's built on a GC, but isn't really compatible with it either.
I'm constantly cleaning up manually in C#, which leaves very little
value in the feature, and a definite tendency to produce unreliability
by users who aren't experts on garbage collection and presume it
should 'just work'.

I support the notion that if the GC isn't removed as a foundational
feature of D, then destructors should probably be removed from D.
That said, I really want my destructors, and would be very upset to
see them go. So... ARC?