Thinktank: CI's, compiler lists, and project automation

[Seb]

On Friday, 2 March 2018 at 06:07:25 UTC, Nick Sabalausky 
(Abscissa) wrote:
> (I'm posting this here instead of D.learn because it isn't 
> really a "How to do XYZ in D?", but rather to invite discussion 
> on high-level solutions to a problem.)
>
> Here's a common problem:
>
> 1. A project (ex, some library) uses travis-ci (and/or other) 
> to ensure compatibility with a range of compiler versions. The 
> travis configuration file includes a manually-curated list of 
> compiler versions.
>
> 2. CI tests are triggered by new commits/merges/PRs in the 
> project.
>
> 3. New compiler versions are released.

Well, just for the record - imho adding your project to the 
project tester is the best way to prevent regressions a priori: 
add your project to the Project Tester (ci.dlang.io).

The Project Tester runs the testsuite of currently ~40 packages 
on every PR at dmd, druntime, phobos, tools and dub
An example: 
https://ci.dlang.io/blue/organizations/jenkins/dlang-org%2Fphobos/detail/PR-6259/2/pipeline

Imho it's te best way as a regression is caught _before_ the 
change is merged and the need for the interaction is on the side 
of the person who would introduce the regression (i.e. the one 
understanding the change).

New projects can be added either by making a PR to the file to 
[1] or pinging me.

[1] 
https://github.com/dlang/ci/blob/master/vars/runPipeline.groovy


> but there are still limitations and downsides (ex: D tester has 
> limited resources

FWIW it's not so limited and highly parallizable. It's already 
configured to automatically create new temporary worker machines 
in the Google Cloud [2] whenever there's more traffic/demand.
Considering the upsides of knowing that your project can _never 
ever_break again is so huge that imho even if more workers are 
needed, it's well-justified as there's quite a net gain.

[2] It currently uses the cheap preemptible VM machines in the 
Google Cloud 
(https://cloud.google.com/compute/docs/instances/preemptible), 
but Martin plans to migrate to Hetzner soon as there are too many 
problems with those VMs.