Discussion Thread: DIP 1028--Make @safe the Default--Final Review

[Joseph Rushton Wakeling]

On Monday, 30 March 2020 at 18:12:03 UTC, Steven Schveighoffer 
wrote:
> Sociomantic avoids unpredictable GC cycles, but doesn't disable 
> it (they still allow collections periodically IIRC). And they 
> are built to be as fast as possible.

There were apps where a single GC cycle was not really an option, 
because even a small GC pause would delay our responding to 
requests where we couldn't afford the delay.  But the solution 
was not really _that_ hard: be strict about using recyclable 
buffers and object pools, preallocate in advance so that there 
would be minimal resizing ... and then be really strict about 
keeping that policy.

There's no reason Discord couldn't have done that with their Go 
app, but if I understood their blog post right, Go's GC 
force-activates a cycle every 2 minutes regardless of whether any 
new allocation has actually happened.  (TBH I do wonder if this 
is really _really_ true, or whether they were just generating 
sufficient garbage to ensure this happened, despite their claims 
of efficiency.)

But in any case Sociomantic could rely on the fact that in a D 
app no new allocations means no chance to trigger a GC cycle 
(which of course is why we preallocated as well as recycling 
buffers and objects: ideally, we wanted no heap allocation after 
app startup).

However, it was only a few apps where this was really necessary.  
In fact I think a lot of the time we were much more strict about 
preallocation and reusable buffers than we needed to be, and the 
strictness was more of a hangover from working around historical 
bugs that occurred when using 32-bit DMD.

Basically, the _other_ problem that arose in Sociomantic's use 
case was that if you want to keep a given app running 
indefinitely on the same server (and there were some apps that we 
never wanted to restart if we didn't absolutely have to for new 
deployments), then you really, really want to be sure that its 
long term memory usage is stable.  A small daily growth can add 
up to a lot over months, and wind up bringing down the app or the 
server.  And in the early days, what they found was that if they 
generated garbage, then slowly, over time, the memory usage would 
creep up and up ... so they instigated this strong "preallocate 
and reuse" policy to work around it.

When I was fairly new in the company I got the chance to 
implement a new app, and quite early on my team lead sat down 
with me to show me how to implement and validate the 
prellocate-and-recycle way of doing things.  The use-case meant 
that it was unlikely there would be a problem if we had a GC 
pause, and we wanted to iterate fast on this app, so I suggested 
we make the code simpler and just rely on the GC.  He explained 
the long-term memory leak issue, but we agreed to let me try and 
observe to see what happened.  And it turned out that no 
garbage-based memory leak emerged.  Which was a nice surprise for 
my lead and all the other old lags in the R&D team.

I don't think anybody ever did work out exactly what the problem 
had been in the early days, but it's likely relevant that by the 
time I broke the rules, the company had been using 64-bit DMD for 
a long time.  IIRC what was suspected (N.B. this is from memory 
and from someone who is not an expert on the internals of the 
GC:-) was that with the 32-bit GC there was something about the 
size of GC pools or memory chunks that meant that it was very 
likely that you could wind up with a chunk of GC memory where all 
of it was in principle recyclable except for a couple of bytes, 
and hence you would allocate new chunks and then the same thing 
would happen with them, and so on until you were using far more 
chunks than should really have been needed.

So, either in 64-bit DMD that didn't happen, or whatever GC bug 
it was had long been fixed anyway.  And once that discovery was 
clearly established, I think we started relaxing the strictness a 
bit in apps that didn't need to care about GC pauses.

The team that grew out of the app I was working on never did have 
to really care about GC issues, but ironically I did wind up 
rewriting that same app to make a lot more use of recyclable 
buffers, though not preallocation.  I don't recall that it was 
ever really _necessary_, though: it was more of a precaution to 
try and ensure the same memory consumption for D1 and D2 builds 
of the same app, given that D2's GC seemed happy to allocate a 
lot more memory for the same "real" levels of use.  Most likely 
D2 just allowed the size of the GC heap to grow a lot more before 
triggering a collection, but we were hyper-cautious about getting 
identical resource usage just on the offchance it might have been 
something nastier.

> That doesn't mean D would beat Rust in a competition on who 
> makes the best discord software. It really depends on a lot of 
> factors, and I don't think generalizing Go and D to be the same 
> because they both have a GC is fair or accurate.

For those apps that really couldn't afford a single GC cycle, we 
did have some discussions about how, if we were writing from 
scratch, Rust's memory model might have been a nice fit (it was 
no fun having to monitor those apps for signs of GC cycles and 
then work out what was causing them).  It would certainly have 
been _interesting_ to try to write those apps in Rust.  But I 
think we would have missed a lot of other things that were also 
important: the friendliness of the code, the ease of iteration, 
and especially the compile-time introspection and metaprogramming 
that even in D1 were a major, major help.

I've had a little bit of a go at metaprogramming in Rust, and ... 
I can't say I like it :-)

It's difficult not to feel that maybe what really made the 
difference for Discord was not really the language, but that this 
time they got the design right.  But maybe, for them, Rust's 
strictness was a way of settling design questions that they could 
have sorted out for themselves but only by having debate and 
consensus and making sure that everyone was consistent in doing 
the right thing.  And Rust probably took all that off the table.