Origins of the D Programming Language

H. S. Teoh hsteoh at quickfur.ath.cx
Sat Dec 1 17:52:16 UTC 2018


On Sat, Dec 01, 2018 at 04:13:20PM +0000, Adam D. Ruppe via Digitalmars-d wrote:
[...]
> The other notable thing though was how all this stuff I think is so
> analogous to the memory safe talk of today. Thread safe by default was
> a huge talking point. For a couple years, everyone was throwing out
> fancy proposals, tons of major stuff was added (including several big
> breakages), the TDPL book was published promising more like `shared`
> meaning something.  We were told this is the future of programming -
> multicore is the big thing, no more serial processors. If D is going
> to go anywhere it MUST capture this. Comparisons with Erlang all
> around.
> 
> But where are we now? Those features are there... but were they the
> revolution? I'd say no. In fact, most people seem to simply ignore
> them and carry on.

IMO, one of the main reasons people ignore these features and carry on
as before is because of the lack of an abstraction that's easy to reason
about and apply in everyday programming.

Thread-safety, with the current abstractions, is exceptionally hard to
reason about, full of hidden gotchas, tricky semantics, and side-effects
that "leak" through the abstractions.  People have difficulty "thinking
in these terms", i.e., using these facilities in a pervasive way,
because it's just so counter-intuitive. It's like being forced to speak
in a foreign language. Some people may be able to throw down the
gauntlet, take the bull by the horns, and learn the idiom despite of the
difficulties, and become masters of it. But the majority of programmers
will simply resort to "learning just enough foreign words to get by"
(the bare minimum constructs needed to make threaded code work), and
revert to a more comfortable idiom (traditional serial programming) for
everything else.

It won't be until an abstraction is invented for threaded computation
that's (1) intuitive to write without feeling like you're walking on a
minefield, and (2) easy enough to reason about that you can reliably
predict the consequences of any arbitrary combination of threaded
constructs, that this kind of programming will be adopted by the general
programmer population.

The recent "nursery" idea from a Python library that somebody posted
here seemed to be a promising step in this direction. But still, it
doesn't seem to be quite there yet.


> And now, memory safety is trumpted as the next big thing that D MUST
> capture. Comparison with Rust all around. (Never mind that D has
> basically already solved memory safety with a well-known,
> widely-implemented traditional solution: conservative garbage
> collection.) Many features being added.
> 
> But where do you think we'll be in five years? I know my guess.

Memory safety is much more than merely having a GC instead of doing
malloc/free by hand.

There's also the pervasive design mistake of C, carried over to C++ and
some other C derivatives, of conflating arrays with pointers (with no
bounds), leading to buffer overruns everywhere.  Having worked with
C/C++ code for more than a decade, I'd hazard a guess that your average
C codebase is probably full of hidden buffer overflows that you just
haven't realized are there yet, and with more looming on the horizon
because it's just so easy to pass the wrong length with the pointer to
your array, or, in bad cases, passing just a bare pointer with no
guarantees that the function receiving the pointer won't write beyond
the end of the buffer.  D solved this major hole from day 1 by making
arrays always carry length around, thus freeing the programmer from the
mental burden of keeping track of array sizes and passing the right
sizes with the right pointers.

These two probably constitute the underlying causes of the majority of
memory unsafety bugs in software. But there are others, like casting
untyped memory (or memory of a different type) to/from typed memory
correctly. This is partially addressed by various small but important
details, like T[] converting to void[] with implicit length adjustment
(implemented in druntime), APIs like std.stdio's rawRead/rawWrite that
use templates to retain type information (with underlying void[]
conversions that benefit from implicit length adjustment), etc.. The
remainder, like returning references to local variables, have still to
be fully addressed by things like the oft-maligned DIP1000.

Essentially, D has already taken care of two of the biggest causes of
memory corruption, and solved a good number of the other causes. But
there are still holes that remain that need to be fixed before we are
100% memory safe.

But if you're expecting a "revolution" from this, don't be surprised at
disappointment, because at the end of the day, what we're doing is
simply to take care of these issues "under the hood", so that your
average D coder doesn't even need to think about memory safety issues
and they can reap its benefits more-or-less implicitly.  It goes along
with Walter's philosophy from airplane design: the intuitive way to
write code will be the correct (memory-safe) way; it will be either
impossible or require actual effort to write memory-unsafe code.  We
want to avoid C++'s mistake, where due to legacy issues the
straightforward way to write code is almost always the wrong way, and
writing correct C++ code requires a lot of effort and care on the part
of the programmer. D's goal is to make the straightforward way correct
by default, and to make incorrect code stick out like a sore thumb.
(Whether it achieved / will achieve this, of course, is a different
discussion.)

So in this sense your bet is right on: in 5 years (and IMO, that's
pretty optimistic) nobody will even notice memory safety because it will
already be there by default, and will be (mostly) implicit, so you reap
the benefits even without necessarily being aware of it. That would be
the sign of our success. But OTOH if in 5 years people are still talking
about memory safe issues in their D code, that's a sign that @safe has
failed.


T

-- 
Живёшь только однажды.


More information about the Digitalmars-d mailing list