allocate array with new

[H. S. Teoh]

On Wed, May 16, 2012 at 02:20:47PM -0400, Jonathan M Davis wrote:
> On Wednesday, May 16, 2012 13:52:12 Steven Schveighoffer wrote:
> > Where the GC gets into trouble is with allocation of large chunks of
> > data.
> > 
> > If the data 'contains pointers', it creates a very good chance that
> > the data keeps some unknown number of blocks from being deallocated.
> > 
> > And just by the sheer fact of how big the data is, it's bound to be
> > pointed at by some piece of stack data, global data, or TLS data.
> > 
> > So you can go a long way by manually managing larger chunks of data.
> > I think it's worth worrying about as you get into large chunks (on
> > the order of 10MB or more).
> > 
> > What really really sucks about this is, as you allocate larger
> > chunks of data, the chances that the GC incorrectly keeps garbage
> > memory get larger. In other words, the more memory you allocate, the
> > less likely it is that the GC will give you some of it back to you!
> 
> Using 64-bit helps with this.

Yeah, IIRC somebody did some tests comparing false pointer incidence in
32-bit vs. 64-bit environments, and found that false pointer incidence
is much lower in 64-bit.

But of course, a large-enough chunk of memory will still have a non-zero
chance of a false pointer into it, which is bad because it will just
keep sticking around.


> > Precise GC will help this *tremendously*, but I'm unsure if we'll
> > ever get to the point of having a fully-precise GC.
> 
> Any improvement to the GC is a welcome improvement.

Didn't Walter say that we're pushing toward precise scanning?

Of course, the fact that we interface directly with C (and have union
types) means that there will always be _some_ place that can't be fully
precise.

But having more precise info, such as which references are slices and
which are free pointers, would help a lot. For example, if the 10MB
chunk is referenced by a slice of 10 bytes, the GC could in theory
release everything except those 10 bytes. You're still out of luck with
free pointers, but in regular D code those should be quite rare.


> > There will always be a need to
> > do manual freeing of memory.
> 
> Yes, but my point is that it's something you do when the need arises
> due to performance concerns in a particular section of code. It's not
> something that you do as a matter of course in your code in general.
> If you're always trying to free memory as soon as you don't need it
> anymore rather than letting the GC do its job (or try to anyway), then
> you might as well be using malloc and free.
[...]

It would be nice if there was an easy way of switching allocators, so
that you can, for example, indicate that all allocations done inside a
particular function should use malloc/free (including, say, AA's,
arrays, etc.), and have the runtime automatically switch back to the GC
on scope exit.


T

-- 
Laissez-faire is a French term commonly interpreted by Conservatives to mean 'lazy fairy,' which is the belief that if governments are lazy enough, the Good Fairy will come down from heaven and do all their work for them.