Openrefine: Unstable Mac CI

[wetneb]

The Travis builds regularly fail because of the Mac sub-task:
https://travis-ci.org/github/OpenRefine/OpenRefine/builds/729523826

This task is currently marked as required, but it is less stable than the builds with "allowed failures" that we currently have. So I would mark the Mac build as "allowed failure" and bring the openjdk12 in the mandatory section.

[thadguidry]

Perhaps we should just move the Mac build to GitHub Actions? I'm not a Mac guy so not the best to help there, but I can certainly try.

[wetneb]

I would not move a single build to GitHub Actions. If we move to that, we should move all runs there.

[tfmorris]

This is the first time I'm hearing that it's been failing. I don't think any of my builds have failed.

Is the failure mode always the same as the example that you linked (looks like the machine failed to boot/initialize)?

The test setup is a sparse array with that build testing not only Mac, but also Oracle JDK 14.

[wetneb]

Here are more examples:

• https://travis-ci.org/github/OpenRefine/OpenRefine/builds/728450315
• https://travis-ci.org/github/OpenRefine/OpenRefine/builds/727971157
• https://travis-ci.org/github/OpenRefine/OpenRefine/builds/725888917
• https://travis-ci.org/github/OpenRefine/OpenRefine/builds/725889319

[tfmorris]

Thanks. It looks like the same failure mode for all 5 examples. This might have a simple solution (retries, increased timeouts) so it seems worth spending a minute or two investigating before just dropping testing on one of our three platforms. I'll volunteer to do this.

Although it looks like the java command is timing out, but I suspect that's a red herring and that it's actually Homebrew installation of databases that is timing out.

[tfmorris]

Travis has finally acknowledged my report, but no progress on fixing it. My current approach is to pursue the migration to travis-ci.com in the hope that it is better behaved.

[tfmorris]

After the migration, the problem exists on travis-ci.com as well, so I'll continue to pursue other solutions.

[thadguidry]

@tfmorris GitHub Actions, GitHub Actions, nudge nudge nudge... :-) https://docs.github.com/en/free-pro-team@latest/actions/reference/specifications-for-github-hosted-runners

[tfmorris]

nudge nudge nudge

Use your words, please. Why do you think migrating to a completely different system like Github Actions is the easiest & cheapest solution to the problem of intermittent Mac build failures? From a brief glance, the configurations for the two systems are entirely different so we'd end up debugging a new CI system from scratch.

[thadguidry]

@tfmorris I thought the determination of the failures was due to resource constraints? If not, and you are still narrowing the problem, then sure, understood and I will stand down and let you focus on narrowing the problem. Sorry.

[tfmorris]

I've allowed the Mac builds to fail for now, but that build also provides our Java 14 coverage, so we need to find a solution for both Java 14 and Mac. I also noticed that we are not testing against the Early Access previews, which we probably should be so that we catch problems before their release.

[tfmorris]

The Mac build is still problematic, but I've substituted a different Java 14 build (and enabled Java 16).

The current build matrix is:

• Java 8 (only) for pushes to feature branches and pull request against master
  For merges to master or pushes to release branches (of the form v3.5xxx) a full matrix of:
• Oracle JDK 8 on Trusty
• OpenJDK 11,12,13,14, 16 on a mix of Bionic & Xenial
• Oracle JDK 14 on Mac (which failed).

[tfmorris]

I reported the problem here: https://travis-ci.community/t/macos-builds-stalling-w-10-min-timeout-before-installing-dependencies-after-cache-install/10013
and they reported that a fix would be deployed by EOD on Oct. 15, but we've had two failures in the last couple of days, so the problem is still occurring (although the build is allowed to fail now).