More fun with toStringz and the GC

[Don Allen]

On Friday, 5 August 2022 at 23:38:22 UTC, Steven Schveighoffer 
wrote:
> On 8/5/22 7:13 PM, jfondren wrote:
>> On Friday, 5 August 2022 at 22:51:07 UTC, Don Allen wrote:
>>> My theory: because gc_protect2 is never referenced, I'm 
>>> guessing that the compiler is optimizing away the storage of 
>>> the returned pointer, the supporting evidence being what I 
>>> said in the previous paragraph. Anyone have a better idea?
>> 
>> A local variable definitely isn't enough: 
>> https://forum.dlang.org/thread/xchnfzvpmxgytqprbosz@forum.dlang.org
>> 
>> This package came of it: 
>> https://code.dlang.org/packages/keepalive
>> 
>
> Yes, but I will warn you, the compilers are smart buggers. I 
> think someone came up with a case where this still doesn't keep 
> it alive (been a while since I made that).
>
> The only true solution is to use `GC.addRoot` on the string and 
> `GC.removeRoot` when you are done.

Steve --

Thanks for this.

But this time I *did* read the documentation, specifically this:
````
Interfacing Garbage Collected Objects With Foreign Code

The garbage collector looks for roots in:

     the static data segment
     the stacks and register contents of each thread
     the TLS (thread-local storage) areas of each thread
     any roots added by core.memory.GC.addRoot() or 
core.memory.GC.addRange()
If the only pointer to an object is held outside of these areas, 
then the collector will miss it and free the memory.

To avoid this from happening, either

     maintain a pointer to the object in an area the collector 
does scan for pointers;
     add a root where a pointer to the object is stored using 
core.memory.GC.addRoot() or core.memory.GC.addRange().
     reallocate and copy the object using the foreign code's 
storage allocator or using the C runtime library's malloc/free.
````

And this, from Section 32.2 of the Language Reference Manual:
````
If pointers to D garbage collector allocated memory are passed to 
C functions, it's critical to ensure that the memory will not be 
collected by the garbage collector before the C function is done 
with it. This is accomplished by:

     Making a copy of the data using core.stdc.stdlib.malloc() and 
passing the copy instead.
     -->Leaving a pointer to it on the stack (as a parameter or 
automatic variable), as the garbage collector will scan the 
stack.<--
     Leaving a pointer to it in the static data segment, as the 
garbage collector will scan the static data segment.
     Registering the pointer with the garbage collector with the 
std.gc.addRoot() or std.gc.addRange() calls.
````
I did what the documentation says and it does not work.

Having a better version of C and C++ with a gc and the ability to 
directly call useful C/C++ libraries is a big D selling point, as 
far as I am concerned. It was a major motivation for the creation 
of Go. But getting the interaction between the GC and foreign 
functions properly documented is essential. Right now, there are 
bits and pieces of advice in the Language Reference, the Feature 
Overview, and the toStringz documentation and none of it tells 
you what you need to know. In fact, it does the opposite, telling 
you to do something (stick a pointer on the stack) that does not 
work, which leads to the "nasty bug" spoken of in the toStringz 
doc. When you waste a lot of a user's time with poor and 
inaccurate documentation, as this did mine, you are not making 
friends. I would advise fixing this asap.

/Don