Build Master: Scheduling II

[Brad Roberts]

On 12/4/13 12:18 AM, Jacob Carlborg wrote:
> On 2013-12-03 20:08, Brad Anderson wrote:
>
>> Historically, Walter has created the release zips (just using the
>> makefile, I believe). As far as I know, work hasn't begun on getting the
>> autotester to roll releases.
>
> I doubt he has a completely automated process of doing it. It has failed too many times.
>
> Having the autotester building and handling everything should be our primary goal.
>

I'm not sure this is really a good goal any more.  The overlap between testing and releasing isn't 
as big as I originally thought.  Add to that that the very definition of what a release is is in 
question.  The only overlap that really exists is that there's a set of boxes that exist and that 
they build in somewhat similar ways.

But there's also a hell of a lot of differences.  Ignoring pulls for the most part, there's little 
coordination between the platforms in terms of what to build.  It's always "whatever is current" 
which changes over time.  There's no current automation for how to package a release that's part of 
the packages being tested (there's some fledgling support inside the packages, there's some tooling 
that lives off in other places).  There's parts of the releases that have no source and aren't part 
of what's being tested.  If we stick to having a zip that contains multiple platforms, there's no 
code to rendezvous multiple platforms together.  The auto-tester covers a smaller set of platforms 
than release builds are currently done for (example, there's several linux distros that have distro 
specific packages, not all of which are represented by test boxes).

All in all, while I was of the opinion that the test fleet could serve the role of release builders 
and pushed that with Walter and Andrei, I'm much less convinced that's the right short term answer 
right now.  It might become the right answer once a number of the open questions are answered and 
once the automation for building the releases are improved, but not yet.  I guess the 'have the 
auto-tester do it' mentality really hand waves away that most of the work still has to be done to 
make it something that might one day be run automatically.

The auto-tester isn't nearly as magic as I think a lot of people think it is.  It's very little more 
than a set of clients that:

0) ask a server for what package/branches to checkout and optionally what package/branch to merge
1) checkout a set of git repos
2) for a pull test: merge a branch in a particular repo
3) run make in each
4) run make unittest in each
5) for each of the actions in 1-4, ship the success or failure status and the log file to the 
results server
6) Repeat forever.

And a server that collects the results and answers the question asked in #0 above.  There's some 
interesting logic in deciding how to answer that question and a little bit of fun logic to sync 
between state between github and the server.  But it's not particularly fancy or complicated at the 
component level.

My current 2 cents,
Brad