understanding Auto-Test

[Seb]

On Thursday, 22 March 2018 at 04:12:00 UTC, John Belmonte wrote:
> On Thursday, 22 March 2018 at 01:51:49 UTC, Mike Franklin wrote:
>> I believe what happened is a different PR was merged.  When a 
>> PR is merged, all tests are invalidated, and the autotester 
>> begins testing them again.
>>
>> There is also a priority affecting which PRs get tested first.
>>  Those that are labeled with "AutoMerge" are given a higher 
>> priority.
>
> I'm still rather puzzled.  My pull request remains with 8 tests 
> pending after several hours.  I can't find any confirmation on 
> the pulls display 
> https://auto-tester.puremagic.com/pulls.ghtml?projectid=1 that 
> it intends to run the tests.   Surely being listed in the "Old 
> results" table doesn't imply pending runs against a new SHA, 
> otherwise why would there be entries there dated 2017?

I give it a shot:

- Every time a PR is merged at dmd, druntime or phobos ALL 
auto-tester results get invalidated
- The auto-tester constantly tries to build PRs which have now 
results based on his queue (it's sorted after merge priority + 
last pushed timestamp)

The traffic is usually so high that your PR will always be at 
pending. That's really not a big deal. On the contrary, once your 
PR has been approved and gets on the merge queue, it will receive 
higher priority and usually will be merged within 1-3 hours.

Also all reviewers are familiar with auto-tester and pending 
tests have zero effect on the reviewers. All the other CIs 
already give a good feedback and if they want to, they can check 
the history of auto-tester for permanent failures

> I can't find any confirmation on the pulls display that it 
> intends to run the tests.

There's none. It _constantly_ reruns your PR.

>  Surely being listed in the "Old results" table doesn't imply 
> pending runs against a new SHA, otherwise why would there be 
> entries there dated 2017?

The timestamp you see on pulls.ghtml?projectid=1 is GitHub's last 
modified timestamp which essentially is timestamp of the last 
push to the respective branch.

And yep there are a few old PRs, but the auto-tester newer tests 
them as its working queue order is sorted by the "Last Commited" 
timestamp.

Did this clarify things for you?