Thinktank: CI's, compiler lists, and project automation

[Nick Sabalausky (Abscissa)]

On 03/04/2018 02:05 AM, Nick Sabalausky (Abscissa) wrote:
> 
> No. That is only sufficient *temporarily*, that is, until the next time 
> the label "dmd" is updated to another new release once again. At *some* 
> point, .travis.yml will still need to be manually updated...
> 
> Scenario 1-2-3:
> 
> 1. Suppose a library supports DMD v2.071.0 and up. Because of this, 
> .travis.yml includes:
> 
>    - dmd  # Ie, dmd-2.079.0, ATM
>    - dmd-2.079.0
>    - dmd-2.078.2
>    - dmd-2.078.1
>    - dmd-2.078.0
>    - dmd-2.077.1
>    - dmd-2.077.0
> 
> Summary: CI now tests 2.076.1 through 2.079.0.
> 
> 2. Release of DMD 2.080.0 occurs. "dmd" now points to 2.080.0 (and 
> forget reproducibility: if 2.080.0 contains an unexpected regression 
> then using travis's "Restart Job" on the "dmd" job will break overall 
> reproducibility.)
> 
> Summary: CI now tests 2.076.1 through 2.080.0.
> 
> So far so good, right? Until...
> 
> 3. Release of DMD 2.081.0 occurs. "dmd" now points to 2.081.0. Unless 
> .travis.yml has been manually updated, it still contains:
> 
>    - dmd  # Ie, dmd-2.081.0, now
>    # WTF?!?! dmd-2.080.0 is no longer being tested!!!!!!!!
>    - dmd-2.079.0
>    - dmd-2.078.2
>    - dmd-2.078.1
>    - dmd-2.078.0
>    - dmd-2.077.1
>    - dmd-2.077.0
> 
> Summary: CI *stops* testing dmd-2.080.0.
> 
> FAIL.

Additionally, what happens when said project above doesn't receive 
commits/merges between steps 1 and 3 (ie, during the time period where 
2.080.0 is the latest release)? Any incompatibilities with 2.080.0 go 
completely unnoticed because CI never ends up testing with 2.080.0.

The "dmd" label does NOT solve the problem.