CI buildbots

[Jonathan Marler]

On Monday, 21 May 2018 at 22:21:34 UTC, Manu wrote:
> On 21 May 2018 at 09:22, Jonathan Marler via Digitalmars-d 
> <digitalmars-d at puremagic.com> wrote:
>> On Monday, 21 May 2018 at 04:46:15 UTC, Manu wrote:
>>>
>>> This CI situation with the DMD/druntime repos is not okay.
>>> It takes ages... **hours** sometimes, for CI to complete.
>>> It's all this 'auto-tester' one, which seems to lock up on 
>>> the last few
>>> tests.
>>>
>>> This makes DMD is a rather unenjoyable project to contribute 
>>> to.
>>> I had a sudden burst of inspiration, but it's very rapidly 
>>> wearing off.
>>
>>
>> It might take hours for CI to complete, but it can take weeks 
>> or months for someone to review your code...so the CI time 
>> doesn't really seem to matter for myself.  That is unless 
>> you're trying to use the CI in your modify/test development 
>> cycle.  However, that's should be solvable by testing locally 
>> in most cases.
>
> I use CI to test the platforms I don't build locally. That's 
> natural for cross-platform development.

Ah I see.  Well for me it seemed worth the effort to setup at 
least a windows and linux machine which covers most testing.  The 
worst was when we were having intermittent seg faults on 32-bit 
OSx...I eventually borrowed my girlfriends macbook to reproduce 
and debug that one...ugh that was a pain.  In any case I'd 
recommend having one posix and one windows platform.  There's 
always VMs if you need em :)

But to address your original concern, I'm not sure that 
decreasing the testing required to integrate changes is 
necessarily a net positive.  If you can decrease test time while 
maintaining the same coverage then by all means, let's do that!  
But I think in general you have to find a balance between the two.

Since you are using CI for your modify/build/test development 
cycle, one idea would be to define some sort of interface that 
the CI's could use to limit what they are testing for a 
particular PR.  You could have it search through the comments for 
something like "limit test to dmd runnable" or something like 
that.