CI: Why Travis & Circle

[Petar]

On Thursday, 14 November 2019 at 17:32:27 UTC, jmh530 wrote:
> On Thursday, 14 November 2019 at 17:06:36 UTC, Andre Pany wrote:
>> [snip]
>>
>> With the public availability of Github Actions I highly 
>> recommend it if you have open source project on Github. If is 
>> free and works well with D and Dub.
>>
>> Kind regards
>> Andre
>
> I'm not that familiar with Github Actions, but I should get 
> more familiar with it.
>
> But my broader question is why both? Don't they both do largely 
> the same things?
>
> I was motivated to ask this by looking at the mir repositories, 
> which have both.
> https://github.com/libmir/mir

Most likely the reason is parallelism. Every CI service offers a 
limited amount of agents that can run in parallel, which limits 
the number of test matrix combinations that you can run in a 
reasonable amount of time. For example, many of the major D 
projects are tested across different OSes and several versions of 
D compilers. Additionally some CIs are faster than others. In my 
experience CircleCI is faster than TravisCI by a large margin.

Another reason is the different types of CI agents. Traditionally 
Travis CI was the first CI service that offered macOS agents for 
free for open-source projects. Now they have experimental Windows 
support as well. On the other hand Travis uses VMs which are 
often noticeably slower to boot than containers. (I think they 
also offered containers for a while, but for some reason they 
deprecated this offering).
CircleCI on the other hand has a really good support for 
containers and in my experience they start in about 1-2s after a 
commit is pushed (whereas VMs can take between anywhere between 
20-60s.

As Andre mentioned, GitHub Actions is a new player in the space, 
which offers an extremely generous amount of compute for public 
repos, courtesy of Microsoft Azure.
(Actually I think they offer exactly the same for public repos as 
what Azure Pipelines offers). However, GitHub Actions has the 
most catch up to do, if you exclude their raw compute power. 
Their UI is extremely bare bones and the configurability is 
lacking behind both CircleCI, SemaphoreCI and even Azure 
Pipelines. However, for most D projects what they offer should be 
a perfect fit.

As for Mir specifically, just check what commands are run by 
circle.yml and .travis.yml and you'll be able to understand 
whether there is actually a need for using multiple CI services:
https://github.com/libmir/mir/blob/master/circle.yml
https://github.com/libmir/mir/blob/master/.travis.yml

I may be wrong, but it seams that they should be able to 
consolidate their testing into just one CI service, but I would 
recommend holding off trying GitHub Actions for now and to go 
with Azure Pipelines, as GitHub's offering is not very stable 
yet. For example two days, due to some change on their side their 
CI completely stopped working on a public repo of mine. After 
researching I found that other people had similar issues (e.g. 
[1]), but fortunately, after contacting their support the issue 
was resolved one day later. In comparison, Circle CI, Semaphore 
CI and Azure Pipelines have been much more rock-solid in my 
experience.

[1]: 
https://github.community/t5/GitHub-Actions/GitHub-Actions-workflows-can-t-be-executed-on-this-repository/td-p/38153