Autotesting dub packages with dmd nightly

[Sebastiaan Koppe via Digitalmars-d-announce]

On Friday, 26 August 2016 at 18:52:17 UTC, Seb wrote:
> That's awesome to know!
>
> How difficult would it be to integrate it with the dlang GitHub 
> PR workflow?

Not at all. Just need an api key from someone with administration 
access. But let's not get ahead of ourselves. Right now I am just 
planning to contact the github api.

Still, there is some work to be done first for purging and 
updating the job queue when prs are updated and probably some 
other cases.

Also there are some choices left regarding the interpretation of 
the results. Right now, for pull requests I do a diff with the 
latest dmd release and collect all the packages that have a 
different outcome. It would be better to run the comparison 
against the commit the pull request was based on, although that 
requires building twice as much.

Currently I am focused on a simple frontend to give people a view 
into the results. It is coming along quite nicely.

> I am just shooting an idea that popped into my head:
>
> We already use CircleCi and Travis for the dlang repos, so if 
> we lock the packages to a fixed version (to prevent failures 
> caused by the package authors), we might be able create a 
> simple file like:
>
> ```
> vibe.d==0.7.29
> mir==0.16.3
> ...
> ```
>
> We could select a subset (e.g. 50-100), s.t. the runtime 
> doesn't get exorbitant.
> Following we could then do enable the checking in CircleCi with 
> sth. similar to:
>
> ```
> wget 
> https://raw.githubusercontent.com/dlang/community-list/master/dlang-packages.master # just an example, idea is to host the file somewhere on GitHub s.t. editing it is easy
> dub fetch your-fancy-tool --version="x.y.y"
> dub run your-fancy-tool --config dlang-stable.packages
> ```
>
> Of course CircleCi doesn't have the access rights to post back 
> to the hook API, but you could send a notification to dlang-bot 
> [1] which has the permissions or let the CI error / fail.
>
> Otherwise you could of course look into setting up your own job 
> queue (or hack with the code from the auto-tester [2]), which 
> might be fun.
I already have my own queue. The important part though is a place 
to keep the results and running queries against it. Currently I 
use various regexes against dub test's output and categorise 
accordingly.

What I am really happy about is the aggressive caching. It allows 
me to build and unit tests a package in 10sec on average.