Leave GC collection to the user of the D library?

[Ali Çehreli]

On 5/9/21 6:51 AM, Vladimir Panteleev wrote:

 > In my experience, calling D from C/C++ works fine as long as 1) the D
 > runtime is allowed to initialize, and 2) all threads which execute D
 > code are registered with the D runtime.

I have the following old branch that tries to fix bugs with attaching 
and detaching:

   https://github.com/dlang/druntime/pull/1989

I also have this worry that if a foreign thread goes without the 
knowledge of D runtime, the program will crash because the runtime would 
try to stop a non-existing thread.

 > If the threads don't need to share state, you could just as well spawn
 > one subprocess per thread, and let it do its own data processing.

That can be achieved by some changes but the issue is not with the 
backend (the daemon); rather, the foreign threads that call the thin 
library layer. This layer does allocate (see below), which may trigger 
collection.

 > Another approach would be to listen on a UNIX socket instead of using a
 > pipe, which allows using `accept` to open new communication channels
 > on-demand.

Aside: I've been under the impression that there couldn't be new pipes 
opened but I learned that a file descriptor can be passed to another 
process over unix domain sockets (only Linux is interesting to me here) 
and the file descriptor and its "passed copy" can be used like a pipe:

 
https://stackoverflow.com/questions/2358684/can-i-share-a-file-descriptor-to-another-process-on-linux-or-are-they-local-to-t

I haven't experimented with it yet but I think I will use it because the 
existing channel I wrote uses File objects of PipeProcess, which places 
large data on shared memory which is re-opened as more space is needed. 
(And thank you for your work in std.process and more! :) )

 > Perhaps it would be simpler to just write the library part in C / C++ /
 > -betterC D.

That's a great idea as well.

 > std.mmfile has many lines, but the work it has to do is
 > actually quite simple.

I know because I've already written the equivalent of mmfile myself by 
accident. :D When things didn't work about 2 years ago I suspected bugs 
in mmfile so I wrote my own wrapper. :)

 > The same is true about std.socket. This will
 > completely avoid your headache with getting the D runtime / GC to play
 > well with the host process's threading model.

Makes sense. I've already turned into someone who uses only D's standard 
library or writes his own. :) The only third party dependency we have is 
cmake-d.

Ali