Adding more projects to the Project Tester

[Jonathan M Davis]

On Thursday, 5 July 2018 21:19:44 MDT Seb via Digitalmars-d wrote:
> So learning from the recent Vibe.d regression fiasco (we
> temporarily disabled a subconfiguration in Vibe.d and promptly
> got a regression in 2.081), I think we should try to add more
> projects to the Project Tester.
>
> The current list is here:
> https://github.com/dlang/ci/blob/master/vars/runPipeline.groovy#L443
>
> Any suggestions?
>
> Why should I add my project to the Project Tester?
> --------------------------------------------------
>
> Once a project is added to the Project Tester, DMD can't regress
> on it anymore as for every PR at dmd, druntime, phobos, tools and
> dub the testsuite of the added projects are run.
>
> How does the Project Tester work?
> ---------------------------------
>
> - By default, it will run the same commands as Travis would do.
> Although, if necessary, custom commands can be used too.
> - It will checkout the latest, stable git tag
>
> Requirements
> ------------
>
> - moderately popular or was prone to regressions in the past
> - rather easy to build (i.e. you don't need to download and
> recompile clang)
> - no flaky testsuite (random errors in the testsuite due to
> network connectivity shouldn't happen. Though there's
> `DETERMINISTIC_HINT=1` set, s.t. you could disable such parts of
> your testsuite)
> - reachable author or development (if there's ever a case where
> we need to push changes via a trivial PR to the repo, it
> shouldn't sit in the queue for weeks)

dxml should probably be on the list. Assuming that this is only on *nix and
not Windows (I have yet to write a test script for Windows), then ./test.sh
will run the tests without -release, and ./testRelease.sh will run the tests
with -release (simply running dub test won't do anything useful, because all
of the unittest blocks are versioned to avoid problems when projects
depending on dxml compile their tests).

- Jonathan M Davis