SAOC Experience Report: Porting a fork-based GC

[Francesco Mecca]

On Monday, 22 July 2019 at 15:58:08 UTC, a11e99z wrote:
> On Monday, 22 July 2019 at 14:03:15 UTC, Mike Parker wrote:
>> Francesco Mecca ha written an experience report for the D Blog 
>> about his SAOC 2018 project, porting Leandro Lucarella's old 
>> GC from D1 to D2.
>>
>> The blog:
>> https://dlang.org/blog/wp-admin/post.php?post=2148&action=edit
>>
>> Reddit:
>> https://www.reddit.com/r/programming/comments/cgdk1r/symmetry_autumn_of_code_experience_report_porting/
>
> interesting idea do Mark phase through fork process.
>
> I want more technical details:
> - about Windows: will such idea work on Windows? does Windows 
> support similar technique? 
> (VirtualQueryEx/ReadProcessMemory/mmap/or_something)
I have no idea about Windows. You can read about it here:
https://rainers.github.io/visuald/druntime/concurrentgc.html

> - can GC-process run full time and just exchange with parent 
> process about changes in memory configuration? or GC-process 
> can not to see changes in parent process after fork? I mean 
> GC-process can see snapshots of parent-process memory in 
> realtime when it has just readonly-access.

It could given that the bitarrays that are needed for the 
algorithm could be mmapped.
In order to have a children running full time we would need a way 
to signal that a new allocation is taking place and to start a 
mark phase.
We could experiment in this direction (it shouldn't be much work 
at the current state) but to me it seems overkill given that it 
is related to the mark routines only and not the whole garbage 
collection cycle.

> - does GC-process uses some AA for building graph of objects or 
> set Mark-bits in situ? in last case COW makes copies so memory 
> differs and GC-process should be forked every GC-cycle.

Last case, but the needed bits could be mmapped in order to be 
shared. Impact is minimal.

> - digits in PR benchmark scary me: max pause 969ms or 260ms. 
> why not to use Golang-GC? Go-GC is non-compacting/non-moving GC 
> that has STW(stop the world) ~10ms and Google can spend money 
> to build best GC that can help to D-users too.

First of all I am not aware of the go-gc being provided as a 
library, it should be tightly coupled to the go runtime.
Also, I think it uses write barriers and D doesn't have them.
Moreover, the go-gc runs in background all the time, trashing the 
cache and stealing cpu cycles. I don't know if the situation has 
changed but that is a major compromise.

Regarding dlang I would suggest to stop worrying about the GC and 
profile. The GC is highly deterministic (a property that few GC 
out there share) so that the user knows when it runs.
Currently there are many facilities for profiling (and more could 
be added) so that bottlenecks caused by the GC can be solved on a 
case by case basis.
Post such as this:
https://blog.cloudflare.com/go-dont-collect-my-garbage/
shows that in real world scenarios people try to tweak the many 
knobs that GC implementers provide in order to adjust overall 
performance. The main difference is that in D you can isolate the 
few cases that are impacted by the GC and avoid garbage 
collection in that path.