[Testing] Using Travis CI (or better alternative) for master/branch/PR testing

[Iain Buclaw via D.gnu]

On 23 June 2015 at 10:15, Marko Anastasov via D.gnu <d.gnu at puremagic.com> wrote:
> On Monday, 22 June 2015 at 08:56:54 UTC, Iain Buclaw wrote:
>>
>> So, I'd be willing to hear of alternatives:
>>
>> https://semaphoreci.com  - However CPU's given are 2, and time to build
>> and run tests is limited to 60 minutes.
>
>
> Hi Iain,
>
> Semaphore cofounder here. The first point is correct, however the 60 minute
> limit applies to single build commands, not the entire build. If you can
> compose your build of n commands, each < 60 minutes, it'll be fine.
>
> I invite you to give Semaphore a try. We'd love your feedback. And I'm here
> for any questions you may have. :)
>

Hi Marko,

Thanks for the update.  Yes, each command step is easily done in 20
minutes.  I just have a few questions which I couldn't find answers
for in the documentation.

1. What kind of storage and memory quotas are in-place on the build environment?

Total storage needed for a building GCC/GDC can be anywhere up to 4GB
depending on multilib/multiarch configuration, or enabled/disabled
features or other languages.  As I am only building specific parts of
GCC, I wouldn't expect the size to ever exceed this.

There are also at least 3 "very big" sources in the GCC backend build
and at least 2 modules in the GDC library build (many more than that
when building unittests) that if built in parallel, their combined
memory usage easily exceeds the 3GB limit on Travis CI.

Infact, depending on the GCC version, a single source file and/or
module one it's own may exceed the memory limit!
https://travis-ci.org/ibuclaw/GDC/builds/67799174#L9590


2. Is the absence of any logging also taken into consideration when
timing out builds?

For instance, the D testsuite run can take anywhere up to 15 minutes
to run depending on number of parallel jobs, and is silent for the
entire duration except in the event that a test fails.  For Travis,
I've had to work around their 10 minute silence limitation by turning
up the verbosity, but that comes with it's own set of problems with
excessive logging.


3. How are caches in Semaphore stored?  Is there a size limit for
files in cache?  Do they expire?

I'd prefer to not download the GCC tarballs from mirrorservice at the
start of each build, which are approaching 100MB in size.  In release
branches, a new minor version of GCC comes out once every 4 months or
so, and in development, a snapshot is released every week, though on
our side, I only bump the snapshot version every month or so.

When investigating Travis, the use of cache is effectively useless
when it comes with dealing with such tarball sizes as it all goes in
S3 storage!

Adding in logic to test if a given tarball is in cache is simple
enough on my end.  Removing old tarballs after I no longer care about
them can be done also, unless your cache servers already take care of
that.


Thanks
Iain.