More radical ideas about gc and reference counting

[Manu via Digitalmars-d]

On 8 May 2014 16:11, Paulo Pinto via Digitalmars-d
<digitalmars-d at puremagic.com> wrote:
> On Wednesday, 7 May 2014 at 20:09:07 UTC, Xavier Bigand wrote:
>
> 4MB?! That is a world of pleasure.
>
> Try to cram a Z80 application into 48 KB. :)

I've never heard of a Z80 program running a tracing GC. (I have used
refcounting on a Z80 though...)

> The main problem nowadays is not automatic memory management, in
> whatever form, be it GC, RC, compiler dataflow, dependent types
> or whatever.
>
> The problem is how many developers code like the memory was
> infinite without pausing a second to think about their data
> structures and algorithms.

Okay, please don't be patronising. Let's assume the programmers you're
talking to aren't incompetent. This discussion's existence should
prove that your theory is incorrect.

Let's consider your argument though, it sounds EXACTLY like the sort
of programmer that would invent and rely on a tracing GC!
The technology practically implies a presumption of infinite (or
significant excess) memory. Such an excess suggests either the
software isn't making full use of the system it's running on, or the
programmers writing the software are like you say.

> Just yesterday I have re-written a Java application that in the
> application hot path does zero allocations on the code under our
> control.
>
> It requires execution analysis tooling, and thinking how to write the said
> code.
> That's it.

Congratulations. But I'm not sure how that helps me?

You know my arguments, how does this example of yours address them?
What were the operating and target hardware requirements of your
project? How much memory did you have? How much memory did you use?
What is the frequency of runtime resource loading/swapping? What was
the pause period and the consequence when you did collect?
Let's also bear in mind that Java's GC is worlds ahead of D's.


I am getting very tired of repeating myself and having my points
basically ignored, or dismissed with something like "my project which
doesn't actually share those requirements works fine" (not that I'm
saying you did that just now; I don't know, you need to tell me more
about your project).
I'd really like to establish as fact or fiction whether tracing GC is
_practically_ incompatible with competitive embedded/realtime
environments where pushing the hardware to the limits is a desire.
(Once upon a time, this was a matter of pride for all software
engineers)
If people can prove that I'm making it all up, and "my concerns are
invalid if I just ...", or whether my points are actually true.

It doesn't matter what awesome GC research is out there if it's
incompatible with D and/or small devices that may or may not have a
robust operating system. D is allegedly a systems language, and while
there is no such definition, my own take is that means it shouldn't be
incompatible with, or discourage certain classes of computers or
software by nature, otherwise it becomes a niche language.

Please, prove me wrong. Show me how tracing collection can satisfy the
basic requirements I've raised on countless prior posts, or practical
workarounds that you would find reasonable if you were to consider
working within those restrictions yourself and still remain compelling
enough to adopt D in your corporation in the first place (implying a
massive risk, and cost in retraining all the staff and retooling).

I don't know how to reconcile the problem with the existing GC, and I
am not happy to sacrifice large parts of the language for it. I've
made the argument before that sacrificing large parts of the language
as a 'work-around' is, in essence, sacrificing practically all
libraries. That is a truly absurd notion; to suggest that anybody
should take advice to sacrifice access to libraries is being
unrealistic.


I refer again to my example from last weekend. I was helping my mates
try and finish up their PS4 release. It turns out, the GC is very
erratic, causing them massive performance problems, and they're super
stressed about this.
Naturally, the first thing I did was scolded them for being stupid
enough to use C# on a game in the first place, but then as I tried to
search for practical options, I started to realise the gravity of the
situation, and I'm really glad I'm not wearing their shoes!
I've made the argument in the past that this is the single most
dangerous class of issue to encounter; one that emerges as a problem
only at the very end of the project and the issues are fundamental and
distributed, beyond practical control. Massive volumes of code are
committed, and no time or budget exists to revisit and repair the work
that was already signed off.
The only practical solution in this emergency situation is to start
cutting assets (read: eliminate your competitive edge against
competition), and try and get the system resource usage down to that
half-ish of the system, as demonstrated from the prior android vs
iphone comparison.

It comes to this; how can a GC ever work in a memory limited
environment? If the frequency of collection is almost 100% of
allocations, then GC must practically be banned everywhere and all the
associated language features lost. How can I rely on library code to
never allocate?
As I see it, this problem is basically synonymous with the current
issue of lazy/unreliable destruction. They would both be solved by
addressing this fundamental issue. And the only solution I know is to
use RC instead.
Eager release of memory is required to make sure the consecutive alloc
has free memory to allocate, problem solved, and destructors work too!
:)