Autotesting dub packages with dmd nightly

[Sebastiaan Koppe via Digitalmars-d-announce]

I have just finished a first iteration of dubster, a test runner 
that runs `dub test` on each package for each dmd release.

see https://github.com/skoppe/dubster

Please provide feedback as it will determine the direction/life 
of this tester.

I am planning on adding a web ui/api next to look around in the 
data.

Today I gave it a spin and let it run on 488 packages on dub 
(about half). The component that runs `dub test` was done on a 
2gb 2vcpu cloud instance. It compiled the packages with dmd 
2.071.2-b2 and it took about 20 sec per package on average.

59 packages didn't build because of missing libraries. See the 
end for a total list of missing libaries.
13 packages caused dmd to use too much memory for the instance I 
was running it on.
112 packages had build errors, almost all of them with exitcode 
1, no segfaults.
35 packages had unittests that retured non zero exitcodes (due to 
exceptions and failing assertions).
213 packages passed their unittests.

The remaining 56 packages I couldn't categorize automatically so 
easily. Would have to take a deeper look at them.

Some issues along the way:

- code.dlang.org has an api but doesn't provide an endpoint to 
retrieve all packages/version. Now I just scrape the site instead 
(thanks Adam for your dom implementation).

- Originally I was planning on running with nightlies, but the 
ones on the download section don't have a git commit hash 
associated with them. For now I just use digger to build the 
latest dmd releases on the worker nodes.

- Some packages when running `dub test` didn't terminate on their 
own.

- Linker errors (a lot of them windows)

aclui, advapi32, asound, blas, bzip2, comctl32, comdlg32, ev, 
fcgi, fdb_c, fmod, ftgl, gccjit, gdi32, git2, GL, gsl, gslcblas, 
gumbo, imm32, iup, iupcontrols, jack, Judy, kernel32, lapack, 
lapacke, leveldb, libco, libshp, lz32, mad, miniupnpc, mpr, mrss, 
mysqlclient, nanomsg, nanovg, netapi32, Netapi32, netcdf, nlopt, 
ole32, oleacc, oleaut32, OpenCL, powrprof, pq, rasapi32, rdkafka, 
rpcns4, Rpcrt4, rpcrt4, sapnwrfc, scrypt, secur32, setupapi, 
shell32, shlwapi, snappy, sodium, tarsnap, tcc, tcl, tcmalloc, 
tk, udis86, usb, user32, version, vfw32, wayland, webp, winhttp, 
wininet, winmm, winspool, Ws2_32, wtsapi32, X11, xcb, xkbcommon, 
zlib, zmq, zookeeper_mt