On Thu, Dec 15, 2011 at 6:43 PM, Brad Roberts <span dir="ltr"><<a href="mailto:braddr@puremagic.com">braddr@puremagic.com</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
On Thu, 15 Dec 2011, Andrew Wiley wrote:<br>
<br>
> On Thu, Dec 15, 2011 at 6:04 PM, Robert Clipsham<br>
> <<a href="mailto:robert@octarineparrot.com">robert@octarineparrot.com</a>> wrote:<br>
> > I really think github needs built in review tools (more advanced than just<br>
> > pull requests) to allow things like the auto-tester to be run, or algorithms<br>
> > like this to be used for manual review and so on.<br>
><br>
> Well, Github does have an API to allow that sort of thing to happen.<br>
> In theory, a bot could examine a pull request, merge it and run tests,<br>
> and post back the results.<br>
<br>
I'm 90% done with adding pull testing to the existing auto-tester fleet.<br>
It's based on the work that Daniel did.<br>
<br>
The basic overview:<br>
<br>
server:<br>
<br>
every 10 minutes, check github for changes to pull requests<br>
(if they had notifications for pull changes, I'd use it instead)<br>
<br>
every time a trunk commit notification is received, check github for<br>
changes to pull requests<br>
<br>
<br>
client:<br>
<br>
forever {<br>
check for trunk build<br>
if yes { build trunk; continue }<br>
check for pull build<br>
if yes { build pull; continue }<br>
sleep 1m<br>
}<br>
<br>
<br>
Left to do:<br>
<br>
1) deploy changes to the tester hosts (it's on 2 already)<br>
<br>
2) finish the ui<br>
<br>
3) trigger pull rebuilds when trunk is updated<br>
<br>
4) add back in support for related pull requests (ie, two pulls that<br>
separately fail but together succeed)<br>
<br>
5) consider adding updating the pull request on github with tester<br>
results. This one needs to be very carefully done to not spam the<br>
report every time a build fails or succeeds.<br>
<br>
6) update the auto-tester grease monkey script to integrate the pull<br>
tester results with github's ui.<br>
<br>
I'll hopefully finish 1 and 2 tonight. I can do 3 manually until it's<br>
automated. I'm not sure about the ordering of 4-6. They're nice to haves<br>
rather than must haves.<br>
<br>
All these extra builds are going to cost a lot of time. There's about 100<br>
open pull requests right now. The fastest runs are on the order of 10<br>
minutes. That's 6 per hour or roughly 17 hours. The slowest are closer<br>
to an hour. So, obviously there's some growing pains to deal with. I'll<br>
probably add a way for github committers to prioritize pull requests so<br>
they build first.<br>
<br>
Luckily this stuff is trivial to throw hardware at.. it's super<br>
parallelizeable. Also the hardware I have for those long runs is super<br>
old. I think the freebsd/32 box is p4 era box. The win/32 is my asus eee<br>
atom based netbook.<br>
<br>
If anyone wants to volunteer build hardware, particularly for the<br>
non-linux platforms, please contact me via email (let's not clutter up the<br>
newsgroup with that chatter).<br>
<br>
Requirements:<br>
I need to be able to access it remotely.<br>
<br>
It needs to be have reliable connectivity (bandwidth is pretty much a<br>
non-issue.. it doesn't need much at all).<br>
<br>
It needs to be hardware you're willing to have hammered fairly hard at<br>
random times.<br>
<br>
I'll almost certainly write some code during the holiday to fire up and<br>
use ec2 nodes for windows and linux builds. With the application of just<br>
a little money, all of those runs could be done fully parallel.. which<br>
would just be sweet to see. Ok, I admit it.. I love working at amazon on<br>
ec2.. and I'm happy to finally have a project that could actually use it.<br>
<br>
Later,<br>
Brad<br>
</blockquote></div><br><div>> <span style>With the application of just </span><span style>a little money</span></div><div><span style><br></span></div><div><font color="#222222" face="arial, sans-serif">Out of curiosity, how much is a little?</font></div>