Beeflang garbage seeker/collector

[H. S. Teoh]

On Thu, Feb 29, 2024 at 03:28:56PM +0100, Timon Gehr via Digitalmars-d wrote:
> On 2/20/24 19:15, Marconi wrote:
> > 
> > Good programmers should have the most control of whats going on in
> > their software.
> 
> So-called "good programmers" have been responsible for awful
> vulnerabilities that ruined people's safety and privacy for decades.
> It's pure hubris to think a non-trivial manually-memory managed
> program has any hope to get manual memory management correct as long
> as only "good programmers" are contributing to it. The fact that you
> are even advocating for this dynamic testing approach rips a big hole
> into this narrative of "good programmers".

The past 3 decades or so of industry experience has proven beyond a
doubt that humans are notoriously bad at manual memory management. Just
read the latest CVEs and note how big a percentage of serious security
vulnerabilities are the result of bugs in manual memory management.
Despite decades of industry experience, this percentage has not dwindled
significantly in codebases that use manual memory management.

In my day job I work with expert C/C++ programmers with decades of
experience under their belts, some of whom exceed my skill level.  We
all know the theory behind how to manage memory correctly.  Guess what?
The very same experts continue making slip-ups and mistakes that cause
memory leaks, pointer bugs, use-after-free, etc.. In spite of decades of
having to fix such bugs and think about how to avoid them.  Clearly,
something isn't adding up.  If decades of field expertise has not
reduced the incidence rate of memory-related bugs, one begins asking
whether the problem isn't so much with "good programmers" as it is with
the fact that *humans are bad at manual memory management*. Conceptually
it's very simple -- allocate once, use n times, free once.  In practice
it's a huge tangled web of complexity that even C experts with decades
of experience have trouble fully mastering.  (Needless to say already
what happens when the fragile codebase falls into the hands of less
experienced coders: expect nothing less than a total sh*tshow.)

And tellingly enough, after about a decade or so in our current project,
the lead programmers have moved away from using malloc/free directly and
have implemented some kind of RC-based system. The fact that this
decision was made tells you that even these experts with decades of
experience realize that manually managing memory is an exercise in
futility; something more automatic is necessary to prevent frequently
repeated slip-ups.  Unfortunately, C being what it is, people still slip
up anyway, like forgetting to call the function that decrements the RC
count once they're done with the pointer, etc..  So the lead programmers
moved toward region allocators -- let the junior programmers make as big
a mess as they want, at the end of the task just free the entire region,
no memory leaks. Problem solved, right?  Nope. Dangling references, use
after free, etc., still abound.

Being C experts, said lead programmers are very down on the idea of GC
(as is usual with people with that background -- including myself at one
time).  But observing all of the above, I really can't help asking, is
GC really *that* bad?  If we'd used a GC language from the beginning,
how many hundreds, maybe even thousands, of man hours would have been
saved from not having to chase down hard-to-catch pointer bugs, memory
leaks, and other such lovely side-effects of mixing humans with manual
memory management.  How much less crashes customers would have seen, and
how much more time and energy we could have spent on implementing
features instead of hunting down pointer bugs.

Of course, if job security is your goal, then by all means, go ahead and
promote more manual memory management.  Me, I wanna make progress in the
problem domain and actually write useful code, not play babysitter with
manually-managed allocations and cleaning up all the leaks afterwards.
I've done my fair share of spending tens even hundreds of hours chasing
down hard-to-find, non-reproducible heisenbugs caused by dangling
pointers, including tracing down the disassembly of the code and
narrowing down the exact conditions to reproduce a rare race condition
(took me a month to find it), all to pinpoint exactly where things went
wrong.  I'm sick and tired of doing this same old thing over and over
again.  Life is too short for manual memory management.  Pass me the GC,
please.

"Good programmers" that always do the Right Thing(tm) don't exist. Even
if they did, 99% of code out there isn't written by them.  That means
99% of code with manual memory management is full of bugs and lurking
security holes waiting to be exploited.  The list of CVEs over the years
prove this beyond any reasonable doubt.  It isn't 1995 anymore; we
programmers, as a profession, ought to do better than still grasping
onto manual memory management when much better options are now
available.


T

-- 
Notwithstanding the eloquent discontent that you have just respectfully expressed at length against my verbal capabilities, I am afraid that I must unfortunately bring it to your attention that I am, in fact, NOT verbose.