GC BlkAttr clarification. Programming in D pages 671, 672. About GC

[Steven Schveighoffer]

On Wednesday, 3 September 2025 at 23:05:45 UTC, H. S. Teoh wrote:
> On Wed, Sep 03, 2025 at 07:56:03PM +0000, Brother Bill via 
> Digitalmars-d-learn wrote:
> [...]
>> C, C++ and D can play shenanigans with pointers, such as 
>> casting them to size_t, which hides them from the GC.
>
> D's current GC is conservative, meaning that any value it sees 
> that looks like it might be a pointer value, will be regarded 
> as a pointer value.
>
> There is an optional precise GC that has been implemented, that 
> can be turned on with compiled-in options or command line 
> options, which uses a slightly less conservative scheme.

The recommendation is avoid *only* storing data in `size_t` that 
points to an allocated block.

Even without the precise collector, the GC has pointer containing 
blocks and no-pointer blocks. this means that it's quite easy to 
accidentally only store a pointer in a `size_t` that will not be 
scanned, even with the conservative GC.

You should only store pointers as `size_t` "if you know what you 
are doing". Otherwise do not do this.

It is fine to make a temporary *copy* of a pointer to a `size_t` 
for example to examine the bits inside. This should leave the 
original pointer alone.

> [...]
>> GC.calloc can allocate memory for a slice of MyClass 
>> instances.  The developer may run GC.free to free the 
>> allocated memory.  But GC may perform its own garbage 
>> collection of GC allocated memory blocks.

`GC.free` is going to free the memory. It will NOT run 
finalizers. It will not collect it again later. I want to make 
that clear.

If you do not explicitly free the memory, and it becomes garbage, 
then the GC will collect it.

As far as a slice of `MyClass` instances, if you mean a slice of 
data that contains the fields of an array of classes, you should 
be very cautious of this. The GC is not equipped to call 
finalizers on such a structure, and so you likely will run into 
lifetime issues.

For classes, I'd just stick with `new`.

For structs, you can quite easily allocate an array of structs, 
and the GC can support finalization of that. Also recommend just 
using `new`.

>> Let's look at each attribute:  (confirm if my analysis is 
>> right,
>> otherwise correct)
>> 
>> FINALIZE - just before GC reclaims the memory, such as with 
>> GC.free,
>>            call destructors, aka finalizers.
>
> This bit is probably best left untouched by user code, and left 
> to the runtime to figure out when/how to use it.

In the latest compiler (2.111), this has been changed to a bit 
that requests finalization upon allocation. The GC uses this bit 
and the typeinfo passed in to determine the correct action. This 
is different from before where the bit was an implementation 
detail that you had to know what you are asking for.

I do agree that you should basically leave this alone. But for 
sure the new treatment of the bit is more robust than before.

Note: changing bits after allocation *does not* take this into 
account, at that point you are modifying implementation details. 
I really would like to get rid of these bits completely and use 
more reliable API (having a set of implementation bits as an 
option is quite dangerous).

>> NO_SCAN - There may be false positives regarding byte values 
>> that look like 'new' allocated pointers.  This can result in 
>> 'garbage' memory not being collected.  If we are CERTAIN that 
>> this memory block doesn't contain any pointers to 'new' 
>> SomeClass allocated memory, then mark as NO_SCAN.
>
> Correct.  Though if you're writing idiomatic D code, you'll 
> almost never need to worry about this.  Whenever you allocate 
> an array whose elements are PODs (without any pointers), the 
> allocator will automatically mark the memory NO_SCAN so that 
> the GC doesn't waste time scanning such blocks.  So things like 
> implicit string allocations will be marked NO_SCAN, etc.  If 
> you're allocating an array or object that contains 
> indirections, then NO_SCAN will not be set, so the GC will scan 
> the interior of suc blocks for pointers to other live objects.

I will add that the concern of scanning non-pointers is pretty 
much obsolete with 64-bit addressing. It's still important to use 
`NO_SCAN`, as it's quite common to allocate large blocks of data 
that are just bytes (e.g. load a file). You don't want to waste 
time scanning that, even if there are no false-positives to be 
found in there.

>>           Question 1: if GC-calloc has allocated MyClass that 
>> has a
>>           string 'name' member, which may expand in size, 
>> would be
>>           still properly apply NO_SCAN.

I would say this is not true. A string has a pointer, it should 
be scanned.

>>           Question 2: if GC-calloc has allocated MyClass, 
>> which may
>>           allocate new MyStudent(...), would that mean 'don't 
>> apply
>>           NO_SCAN'?
>
> It's very simple.  If a memory block may contain pointers, then 
> it should not be NO_SCAN.  If a memory block never contains any 
> pointers, then it can (should) be marked NO_SCAN.

100% correct.

> Normal D code does not need to fiddle with GC flags.

Great advice!

>> NO_MOVE - For GC.realloc, if increasing memory allocated, and 
>> it's not available, throw 'MEMORY_NOT_AVAILABLE' exception.
>
> Correct. You might want to use this flag if you have non-D code 
> that might be holding pointers to this memory block, e.g., if 
> you passed a pointer to some D array to C code which retains it 
> in some C-managed pointer, and the C code expects the array to 
> still be there later.
>
> It's not very often that such situations come up, though.  When 
> passing GC-allocated data to C code, it's generally a good idea 
> to keep a reference to it inside D code so that the GC can find 
> the reference anyway.  Since D doesn't have a moving GC, this 
> is really all you need to do.  Again, unless you're doing 
> something unusual, you probably don't need to touch the NO_MOVE 
> flag.

No, this is not correct. `NO_MOVE` is supposed to mean that a 
moving GC cannot move this block (and fix up pointers to it).

Given that we have a conservative GC, which scans the stack 
conservatively *including C stacks*, and we will always have one, 
I would say this bit should just be deprecated.

Indeed, it is completely ignored in the current GC.

>> APPENDABLE - For D internal runtime use.  Don't mark this 
>> yourself.
>
> Yes.

Also improved with D 2.111. The `APPENDABLE` bit is now an input 
to malloc that tells the GC this is an array (including adjusting 
the size to deal with padding space). The GC now handles array 
runtime features directly, and so it understands what this means.

So in fact, this is a bit you can set, and there are currently 
unexposed GC interface functions that can be used to manage the 
array. They have not yet been exposed in `core.memory`, because 
we are not sure if these are the final interfaces we want.

However, *allocating* an array with this bit will do exactly what 
you expect (and managing the resulting array with the normal 
array management functions such as appending or `capacity` will 
work).

I do still recommend using `new`.

>> NO_INTERIOR - This says that only the base address of the 
>> block may be a target address of other GC allocated pointers.  
>> All other possible pointers are 'false' pointers.

Yes, though I would say it like:

"only pointers found while scanning that point to the exact 
target address may be considered pointers to the block."

Again, this is really only of great use in 32-bit addressing.

>> Perhaps I am missing the fundamentals of various D garbage 
>> collectors.
> [...]
>
> The various GC flags are simply hints that let you influence 
> the scanning process to some extent. The NO_SCAN bit means that 
> upon reaching this block, don't bother scanning its contents to 
> find more pointers (because there are none). The NO_INTERIOR 
> bit means that if the GC finds a pointer-like value that looks 
> like it points to the inside of this block, ignore it as a 
> non-pointer, because pointers to this block only ever point to 
> its head (the supposed pointer is actually not a real pointer, 
> but an integer value that happens to have a pointer-like value).
>
> The other flags have very specific uses that, if you don't know 
> what they actually do, you probably don't need them and 
> shouldn't touch them.

Flags you should be able to use:

* `NO_SCAN`
* `FINALIZE`
* `APPENDABLE`
* `NO_INTERIOR` (very cautiously)

Do not use any other bits directly. A future version of D likely 
will migrate these into function parameters instead of providing 
bits.

-Steve