[RFC] Closing pull requests while a new build is starting causes failures

[maldoinc]

Build failures often indicate a fault with the software being built such as compilation or test failures. However, there is a subset of build failures (called environmental or non-verification failures) that are not expected within a standard application development lifecycle, such as build configuration failures, dependency resolution failures, infrastructure failures, and so on.

Problem: One type of environmental failure is due to due to git references being no longer available at the time a build job starts. For example, pull requests are merged shortly before the start of the scheduled build, causing a build failure due to an unfound (because merged) branch. In your project, we detected 1 failure of this type over the May-June timespan. For instance, we found that Build #26 did not pass due to the above issue.

Solution: It is generally recommended to wait for the build results before merging a pull request. If you wish to skip builds you may also use the SKIP CI tag in your commit message. Additionally, it is also possible to skip builds containing non-code changes as shown in this example and below:

+ before_install:
+  - |
+      if ! git diff --name-only $TRAVIS_COMMIT_RANGE | grep -qvE '(.md)|(.png)|(.pdf)|(.html)|^(LICENSE)|^(docs)'
+      then
+        echo "Only doc files were updated, not running the CI."
+        exit
+      fi

---

Disclaimer: I developed a tool that repairs environmental build failures and I am now evaluating its usefulness for open-source projects.

Please up/downvote the issue to indicate whether you agree/disagree with the report and the proposed fix.