Ref counting for CTFE?

[Jonathan M Davis via Digitalmars-d]

On Thu, 29 May 2014 11:22:54 -0400
Steven Schveighoffer via Digitalmars-d <digitalmars-d at puremagic.com>
wrote:

> One subject that frequented the talks at dconf was the poor
> performance of CTFE and mixins.
>
> The major issue as I understand it (maybe I'm wrong) is the vast
> amounts of memory the compiler consumes while building mixin strings.
> In fact, one of the talks (can't remember which one) mentioned that
> someone had to build their project in steps so the compiler did not
> crash from OOM.
>
> In CTFE, we are not constrained by the runtime GC, and in fact, we
> have no GC at this point (it's disabled). What about implementing
> rudimentary, possibly slow but correct, reference counting for CTFE
> allocated data? It doesn't have to be perfect, but something that
> prevents consumption of GB of memory to compile a project may make
> the difference between actually compiling a project and not. It would
> also be a nice little confined environment to try out ref counting +
> GC for cycles.

That might help, but the core problem with CTFE (as Don explains it) is that
currently each value sits on the heap, and when you mutate it, you get a whole
new object allocated on the heap. So, something like

int i = 0;
while(i < 10)
    ++i;

would be allocating a value for i on the heap 10 times. e.g. something like

int* i = new int(0);
while(i < 10)
    i = new int(*i + 1);

So, you end up with an insane number of allocations for basic stuff. CTFE was
originally pretty much a hack in the compiler, so it was a huge mess. Don went
to a lot of time and effort to clean it up so that it actually has a single
entry point in the compiler instead of being scattered throughout the compiler
in hard-to-find places. All of that had to be done _before_ performance
improvements could even be explored. Unfortunately, after Don got to that
point last year, he didn't have time to continue working on it, and no one
else has picked up the torch (I expect that he'll be back to it, but I don't
know when). Don is convinced that simply making it so that CTFE has true
mutation for _integers_ (without even doing it for anything else yet) would
result in enormous speed gains (and it would obviously significantly reduce
the memory requirements in the process).

So, it looks like there are fundamental issues with CTFE that really should be
solved before we discuss stuff like reference counting its memory. And just
solving those could make referencing counting irrelevant.

- Jonathan M Davis