[dmd-internals] generating pull requests for DMD

[Brad Roberts]

On 5/29/2011 4:07 PM, Robert Clipsham wrote:
> On 29 May 2011 19:13, Walter Bright <walter at digitalmars.com <mailto:walter at digitalmars.com>> wrote:
> 
>     For those doing compiler patches, I know it's a tough job, and thanks!
> 
>     But please consider running the dmd test suite (and try to build/run phobos unittests) before issuing a pull
>     request. There are a lot of interdependencies in the compiler, and the test suite is designed to flush them out. The
>     cool thing is the test suite consists of cases that are already minimized! The suite also is designed to not take
>     that long to run.
> 
>     I run it constantly when I do dev on the compiler, and I constantly get dinged by it for something I overlooked. I
>     consider the D test suite to be a major asset for us.
> 
>     Please use it. If you've got problems using it, please post here and we'll try to help.
> 
> 
> An idea relayed from Vladimir Panteleev: It could be a good idea to integrate pull requests with the automatic test
> runner - github has APIs for pull requests, the test runner could pull out the commits, test them, then post a comment
> on the pull request.
> 
> I see a few problems with this however:
>  - Added load for the test runner (although, they could be run with low priority and only when there aren't commits to
> master to test)
>  - Security - if you're testing all pull requests some sort of sandbox would be needed so the patch isn't running rm -rf
> or something more malicious
>  - More commits can be added to pull requests later on, it would need re-running in this case
>  - The test suite could be altered to make it look like the pull request is fine - although I guess anything that gets
> integrated will be looked over anyway, so this isn't much of an issue.
>  
> -- 
> Robert
> http://octarineparrot.com/

All of that could be dealt with, probably, but it still doesn't fix the underlying issue: developers should be running
the tests on the code they ask to be pulled themselves, BEFORE the pull request is filed.  Not doing so puts potentially
buggy code in the face of others to soon.

Automated testing could help with finding conflicts between pull requests or something ahead of it being merged.  But
using it as a crutch to not test yourself isn't something I want to encourage.