Disadvantages of building a compiler and library on top of a specific memory management scheme

[Frustrated]

Any time you hard code code something you are also creating
constraints.  If people build something on top of what you have
built they inherit those constrains.

Take building a house, once the foundation is laid and the
framework is added it becomes next to impossible to go back and
modify the foundation.

One big problem with D is that the both the runtime and library
are hard coded to use AGC. Switching these to use ARC or MMM are
just building different foundations but all creating very serious
constraints on the the users of D.

Because the runtime and library are created and maintained by
some dedicated and smart people, the "simplistic" way of memory
management does not need to be taken. (I realize it seems like
more work but it isn't given the n-fold benefit it creates for
all those that use it)

So, why not create a design pattern that allows one, at compile
time, to use any specific memory management technique they want
for the runtime and library(and user code)?

e.g., switches could be used to recompile the source when a
different pattern is desired or libraries could be created that
use different methods. e.g., phobos64GC.lib.

To make any of this work, and work efficiently, one must abstract
what is common between all memory management techniques.

Since allocation is always explicitly defined, it is easy to
handle. Which every  memory management technique is used, it
would be called/notified on allocation. The deallocation is the
tricky part. (reallocation is another potential issue)

For manual memory management the deallocation is explicitly
called by the user. This in and of itself is significant and
either allows the compiler to know that manual memory management
is used or to replace it with automatic management if desired.

e.g.,

{
auto a = new!strategy A;

release a; // <- explicit deallocation: replaced with
strategy.explicitDeallocation(a);
}

vs

{
auto a = new!strategy A;

// implicit deallocation: compiler inserts
strategy.scopeDeallocation(a);
}

implicit deallocation could necessarily be handled by the GC.
Basically if the programmer forgets to deallocate a type then it
is automatically handled(by say AGC or ARC or whatever). The
programmer, of course, should be allowed to deal with how this
actually behaves, such as make using a weak reference.

The key point here is that if we abstract the memory allocation
and deallocation schemes then many memory management techniques
should be possible(and possibly simultaneously with no hard coded
reliance on any specific technique.

Both manual and automatic schemes could be used simultaneously in
that if the user fails to release memory explicitly then
automatic releasing is used(AGC and/or ARC). This allows more
predicable memory management and in a perfect world, if the
programmer always properly released memory then no automatic
collections would occur. In the real world the programmer could
disable automatic collections and create weak references or use
automatic collections only on the debug build and log when they
occur to fix(e.g., he forgot to create a weak reference or
release memory explicitly). This could occur on production builds
too, logged, and reported. (so the production code might not be
ideal performance wise but be fixed in future builds).

Different allocation strategies have different ways have handling
how they deal with explicit and implicit calls. One strategy
could report all cases of implicit scope deallocation for the
programmer to look at. Another could completely ignore any
implicit deallocations which would turn off any automatic
collections = most performant but least safe.


One could define a global strategy and then local strategies
e.g., new A uses global strategy and new!myMMM A uses myMMM
strategy for the variable A. The interactions of strategies would
be relatively easy I think. If say, we have

class A
{
     B b;
     this() { b = new!strategy Y B; }
}

auto a = new!strategyX A;

Then on release of A, strategy X would call strategy Y's
deallocation for B. If it is "non-deterministic" then a
notification would be setup so that when b is actually released a
can be fully released. In some cases this could occur immediately
but is meant more for circular references.

e.g., if we have

class B
{
      A aa;
}

auto b = new!strategyZ B;
b.aa = a;

when we free a, b is subjected to freeing, which involves freeing
aa, which involves freeing a. Since a is already in the state of
being free'ed, it can be completely free'ed.

Of course, such a system might be slow if not properly designed
or may just be inherently slow.


Regardless, it is a strategy and other strategies might not work
this way(but there would still need to be some commonality
between them and it would be up to the compiler to optimize
things as much as possible.

The benefit of such a method is that one could test various
performance of different strategies and even, say, attempt to
move one automatic management(implicit deallocations) to explicit
to improve performance.

Basically the compiler could insert implicit deallocations
anywhere it things they are suppose to go... e.g., for local
variables that never have outside references. The allocation
strategy could do nothing(e.g., AGC) or immediately free(e.g.,
malloc) or reference count(e.g., ARC) or even throw an
error(e.g., MMM: "oops! I forget to deallocate or use @weak).

But what's great, is that by using a pattern for memory
allocation and deallocation, regardless if the above method is
the basis or not, D and phobos are no longer hard coded to one
scheme or another and a proper foundation is laid instead one
that is inflexible and limiting. There would no longer be
arguments about which memory management method is best and more
time with getting things done. Also, it would be very appealing
to the outside world because, say, they need to write some code
for the pic16 and need a very specific way of handling memory...
they'll much more likely be able to do it efficiently in D than
any other language... since not only does D offer complete
abstinence of built in management, it would also allow
"strategies" to be used.

Anyways, just an idea... hopefully good enough to get the point
across that we need a better way and we all can stop arguing over
specific details that are somewhat moot.