[Dlang-internal] dmd semver experiment

[Johan Engelen]

On Thursday, 8 March 2018 at 18:25:44 UTC, Martin Nowak wrote:
> On Wednesday, 1 November 2017 at 10:48:15 UTC, Martin Nowak 
> wrote:
>> v8.0.0 - Jan 1 2018
>> v8.0.x - unscheduled patch releases
>> v8.1.0 - Mar 1 2018
>> v8.1.x - unscheduled patch releases
>> v8.2.0 - May 1 2018
>> v8.2.x - unscheduled patch releases
>> v9.0.0 - Jul 1 2018
>> v9.0.x - unscheduled patch releases
>> v9.1.0 - Sep 1 2018
>> v9.1.x - unscheduled patch releases
>> v9.2.0 - Nov 1 2018
>> v9.2.x - unscheduled patch releases
>> v10.0.0 - Jan 1 2019
>> v10.0.x - unscheduled patch releases
>
> Would make sense to release previews of the upcoming major 
> release, maybe every other month to stick with todays pace.

Hi Martin and others,
   I meant to discuss the current release pace at DConf, but 
unfortunately I didn't, so writing here instead.

I feel that the current release pace of DMD is (way) too 
frequent. I am referring to the pace of compiler and language 
changes, as opposed to releases with only bug fixes. Currently, 
there is no separation of the two. Because DMD's releases are not 
tested well [1], this results in there not being high quality 
releases of DMD: each new release is a combination of improved 
quality (bug fixes) and reduced quality (new features / language 
changes / regressions). From my personal experience, this leads 
to there not being a good version to upgrade to. Whichever 
version is chosen, it always comes with improvements _and_ 
regressions.

As an unfortunate side-effect of DMD's release pace, LDC has also 
been releasing at breakneck speed, in order to provide an LDC 
version for every major dlang version (e.g 2.077, 2.078, ...). 
Traditionally LDC releases are based on patched DMD releases 
(e.g. 2.079.1), sometimes including backporting some fixes from 
more recent frontend versions. The patched DMD releases are more 
tested (and fixed) and they thus better meet the higher quality 
standard that is desirable for LDC as an optimizing production 
compiler. However, with the current pace, there is simply no time 
for any real testing any more. Nor is there time to incorporate 
(most) fixes to issues found on released versions in patched 
versions, because by the time issues are found and fixed there is 
already a new major version out (backporting does happen, but, 
understandably, not sufficiently).

To illustrate the problem, I can share the experience of 
upgrading to newer compilers at Weka.
I have yet to experience a compiler upgrade without compilation 
breakage and without linking breakage, and there have been very 
serious execution breakage issues too [2]. Fixing the code (e.g. 
accepts-invalid bugs, deprecations), adding workarounds 
(regressions, language changes), tracking down compiler or Phobos 
or code bugs, etc., all take quite some time meaning that 
actually _testing_ the new compiler only starts after several 
weeks. This is an optimistic estimate; currently Weka is at dlang 
2.076 (LDC 1.6.0) and we now start real testing of dlang 2.078 
(LDC 1.8.0, released over 2 months ago). As you can see, Weka is 
skipping versions, mainly because of the time it takes to upgrade 
the compiler is longer than the period between compiler releases.
Recent events [2] made people at Weka and me much more wary of 
compiler updates, and perhaps the testing phase will also take 
longer than before.


I feel currently the project is racing at too high speed pumping 
out releases, at the cost of quality and carefulness that I feel 
is appropriate for compiler/language development (adding to 
existing quality-of-implementation problems).
Thus I am very much in favor of major releases every half year 
(or longer), with only minor releases in-between that do not add 
new features or language changes.

Thanks for your consideration.

Kind regards,
   Johan

[1] the CI testsuites are not at all exhaustive (frequently 
frustrating LDC development), and I am not aware of large 
industrial projects participating in testing DMD releases.
[2] struct layout differences between LDC 1.2.0 and 1.6.0 were 
causing very severe issues, and we actually put in effort trying 
to revert to LDC 1.2.0 again! Luckily, the revert didn't happen 
and Weka is still on 1.6.0.