Thoughts on Backward Compatibility

[Carl Sturtivant]

On Friday, 16 February 2024 at 04:38:03 UTC, H. S. Teoh wrote:
> On Fri, Feb 16, 2024 at 01:44:51AM +0000, Paul Backus via 
> Digitalmars-d wrote: [...]
>> ### What can we learn from this for D?
>> 
>> First, that the success and popularity of a programming 
>> language is mostly determined by factors other than stability 
>> and backward compatibility (or lack thereof).
>
> +100.  Over the past 5-10 years or so, I've been finding myself 
> wishing that D would introduce some breaking changes so that it 
> could clean up some of its dark, ugly corners that have been 
> like flies in the ointment for a long time.
>
> At the same time, I'd had deprecations and breaking changes 
> that are really rather minor, but extremely frustrating, 
> because:
>
>
>> Second, that even without an edition bump, small-scale 
>> breaking changes with easy migration paths aren't a big deal.
>
> The worst feeling is when you upgrade your compiler, and 
> suddenly you find yourself having to do major code surgery in 
> order to make previously-fine code work again.  Having an easy 
> migration path for breaking changes is very important.
>
> I'd also add that the migration path should be *easy*: it 
> shouldn't take too much thought to upgrade the code, and should 
> not involve tricky decisions based on subtle semantic 
> differences that require deep understanding of the code to make 
> the right choice.
>
> The std.math.approxEqual deprecation is a major example that I 
> keep running into.  It's intended to be replaced by isClose, 
> with all the right intentions. But it was frustrating because 
> (1) it didn't feel necessary -- previous code worked fine even 
> if there were some pathological cases that weren't being 
> handled correctly.  (2) The deprecation message didn't give a 
> clear migration path -- isClose has different parameters with 
> subtly different semantics from approxEqual, and it wasn't 
> obvious how you should replace calls to approxEqual with 
> equivalent calls to isClose.  There were also no easy defaults 
> that you could use that replicated the previous behaviour; you 
> had to sit down and think about each call, then look up the 
> docs to be sure.  (3) The choice of name felt like a serious 
> blunder, even if it was made for all the right reasons.
>
> All of these added up to a very frustrating experience, even if 
> the intentions were right.
>
> If we could have done this over again, I'd have proposed to 
> keep the old semantics of approxEqual, perhaps add another 
> parameter that would use the new semantics. And make sure the 
> deprecation message is clear about how exactly you go about 
> deciding what to put in the new parameter. I.e., for lazy 
> authors not changing anything would let their code continue to 
> work as before; if they wanted to have the new semantics they'd 
> have to explicitly opt in.
>
>
>> Third, that even with an edition bump, large-scale breaking 
>> changes that make migration difficult should probably be 
>> avoided.
>
> Yes, large breaking changes are a no-no.  Unless my old code 
> can continue compiling as before, and I have to opt-in to the 
> new stuff. Editions would help with this, but it still depends 
> on the execution. There should always be a good migration path 
> that doesn't require you to rewrite 5-10 year old code that you 
> no longer remember the details of and can no longer confidently 
> reimplement without spending disproportionate amounts of time 
> to re-learn the ins and outs of it.
>
>
>> Fourth, that breaking changes should be used to give D 
>> programmers more of what they already like about D, not to 
>> take the D language in new directions.
>
> TBH, @nogc, dip1000, @live, etc., feel a lot like D trying to 
> go in entirely new directions.  The fact that it's been years 
> and still practically nobody understands exactly how it works 
> and what it does, is not a good sign. And all this while things 
> like `share` and static initialization of AA's are stagnating.  
> Built-in AA's are one of my major reasons for choosing D, and 
> seeing it languish for years with elementary features like 
> static initialization not fixed is quite disheartening.  Worse 
> when it feels like D wants to move to newer pastures when its 
> current features are still half-done and has problematic corner 
> cases.  I.e., what I like about D is stagnating, while new 
> features that I have little interest in are being pushed on me.
>
>
>> To Walter, Atila, and the rest of D's leadership, I hope this 
>> post provides some helpful data points for you to take into 
>> account when designing D's language editions and planning 
>> future language changes.
>> 
>> To everyone else reading this, I'd like to leave you with one 
>> last question: what do **you** like about D? What strengths 
>> does D have, as a language, that you'd like to see become even 
>> stronger?
> [...]
>
> What I like about D:
>
> - Meta-programming power.
>    - CTFE should be improved.  By a lot.  It was a big 
> disappointment
>      that Stefan's newCTFE never materialized.  IMO we should be
>      improving this story instead of trying to chase rainbows 
> like ARC
>      with @live and dip1000 and what-not.  We should make this 
> so good
>      that I'll never need to use an external codegen utility 
> again. And
>      it should not introduce crazy compile times. This is a 
> primary D
>      strength, its story should be maximally optimized.
>
>    - The template story should be improved.  There should be a 
> way of
>      working with templates that cut down on needless bloat.  
> Lots of
>      room for exploration here.  We shouldn't be confined by C++
>      limitations here. This is one of D's primary strengths and 
> where we
>      can pioneer even more.  One area is improving IFTI to make 
> it work
>      for even more common cases.  Another is recognizing common 
> patterns
>      like chains of ranges, and optimizing symbol generation so 
> that you
>      don't end up with unreasonably huge symbols. Esp. when 
> it's a
>      one-of-a-kind UFCS chain (it's unlikely you're ever going 
> to have
>      exactly the same chain twice with exactly the same template
>      arguments -- no point encoding every argument in the 
> symbol, just
>      an ID that gets incremented per instantiation is good 
> enough).
>
>    - Compile-time introspection and DbI. This is another huge D
>      strength, and we should be working on streamlining it even 
> more.
>       - Clean up __traits(), make std.traits more sensible.
>       - Fix things like scoping issues with static foreach.  
> Introduce
>         local aliases so that static foreach doesn't need crazy 
> hacks
>         with {{...}} and temporary templates just for injecting 
> new
>         identifiers per iteration without running into multiple
>         declaration errors.
>       - Improve the syntax for retrieving members of some 
> symbol.
>         Something prettier than __traits(getAllMembers,...). 
> This is a
>         primary D strength, it should be dressed in better 
> syntax than
>         this.
>       - Maybe first-class types to make the metaprogramming 
> story even
>         more powerful.
>
> - GC.  Instead of bending over backwards trying to woo the 
> @nogc crowd
>   who are mass migrating to Rust anyway, what about introducing 
> write
>   barriers that would allow us existing D users to use a much 
> more
>   competitive GC algorithm?  Stop-the-world GC in 2024 
> shouldn't even be
>   a thing anymore. We aren't in 1998 anymore.  D should embrace 
> the GC,
>   not sacrifice it for the sake of wooing a crowd that isn't 
> likely to
>   adopt D regardless.  Instead of trying to get away from the 
> GC, what
>   about making the GC experience better for existing D users?
>
> - Built-in AA's.  It's been at least a decade.  Why is static
>   initialization support still sketchy?
>
> - Built-in unittests. The default experience should be 
> top-of-the-line.
>   We shouldn't need to import a dub package for something 
> beyond the
>   current dumb built-in test runner. Named unittests, the 
> ability to
>   select which tests to run, the ability to run all tests 
> regardless of
>   failure and show stats afterwards -- these are all basic
>   functionalities that ought to work out-of-the-box.
>
> - Automatic type & attribute inference. `auto` was 
> revolutionary when I
>   first joined D (C++ got it only years later).  We should 
> improve type
>   and attribute inference to the max (e.g., in default 
> parameters for
>   enums: there should be no need to repeat the enum name). 
> Nobody like
>   spelling out attribute soup, just like nobody likes spelling 
> out
>   explicit types when it's already obvious from context. The 
> compiler
>   should automate this to the max.  A little bit of breakage 
> here IMO is
>   acceptable as long as it gets us to an even better place.  
> Have
>   negated attributes be a thing as well.  In fact, make 
> attributes
>   first-class citizens so that we can use DbI / metaprogramming 
> to
>   manipulate it.  This is what we should be focusing our 
> efforts on
>   instead of trying to woo an amorphous group of hypothetical 
> potential
>   users somewhere out there who aren't particularly likely to 
> adopt D to
>   begin with.

+100
And the subtle business mentioned first of cleaning up dark ugly 
corners with breaking changes gets +110. Here are a couple of 
bottom level ones that jumped out at me in the last few days.

--- Fix void to be a type like any other and not an unconsidered 
edge case to trip over.

--- Remove automatic conversions between signed and unsigned so 
that unconsidered bizarre semantics is excluded from D.

Here's a sane take on GC, not more ideology.
https://bitbashing.io/gc-for-systems-programmers.html
--- Make the GC a 21st century GC! And acknowledge that using it 
by default makes sense as per the short article.