Ddev: arm64 cleanup tasks: Exclude mysql, old mariadb versions, run tests

[rfay]

Describe the bug

DDEV-Local now supports linux/arm64 and the binaries are provided, but:

• [x] Only mariadb 10.1+ is supported because there are no packages/upstream docker images for mariadb <10.1 or mysql (any version). But ddev doesn't know this.
• [x] Tests don't know not to run with those values (Fixed in #2568)
• [x] Tests are not yet run on arm64 (fixed in #2568)

/cc @dennisameling

[dennisameling]

I was actually about to create an issue "future of multi-arch on DDEV", so this is good timing 😉 Wanted to mostly focus on the test/build part, so here's my two cents:

Running tests in PRs/commits

• CircleCI doesn't support hosted ARM64 agents yet ❌
• GitHub Actions doesn't support hosted ARM64 agents yet ❌ (the self-hosted runner supports it, but that'd involve setting up your own AWS ARM64 VM and installing the self-hosted runner on it)
• Travis CI supports hosted ARM64 agents ✅ (just learned about that!)

For running tests on arm64, would you be OK with going for Travis, only for the arm64 part?

Building + testing images on the native architectures (amd64 + arm64) would be best IMO in terms of performance. You could run arm64 tests on amd64 through QEMU, but it will be very slow.

Like you mentioned in the issue description, tests will first need to know what (not) to test on arm64.

Releasing with Git tags

In terms of pushing the Docker images, I assume that you want to add that to CI in the future as well, instead of having to push manually/locally? AFAIK docker buildx doesn't support building images on different architectures and pushing them to Docker Hub separately (e.g. an amd64 machine would push ddev-webserver-test-20201009, then an arm64 machine pushes ddev-webserver-test-20201009 as well. The arm64 version will just overwrite the amd64 version in my testing). So, an amd64 machine will need to do cross-compilation (like we're already doing in the Makefiles now) and push to Docker Hub for both archs at the same time.

The situation would then look something like this:

I probably forgot some crucial aspects here, but these are some first brain dumps. 😊

[rfay]

Using TravisCI sounds fantastic to me. I've been experimenting with this on my Raspberry Pi, but that doesn't scale well and would probably be too slow.

The first step is to be able to build on arm64 though. Currently the Makefile can't build on arm64 because it depends on a docker build strategy, and the drud/golang-build-container doesn't support arm64. I can think of two solutions right now:

1. Change Makefile so that the docker-based build is optional, perhaps driven by environment variable. Use native golang for building. But requires that the static tools be installed locally.
2. Build an arm64 version of drud/golang-build-container as you've done with all the other images. This is probably the most consistent path forward.

The docker-based build technique is historical, and it's slower than native build by a bit. So having a native build option isn't a bad solution. It was only 2-4 years ago though that golang builds were absolutely dominated by "it works on my machine", problems so having a "build it the same way everywhere" solution as we have with docker was fundamental. I still like it and like the fact that our CircleCI builds use the same docker image as our Windows builds. But I'm annoyed by how much slower it is.

You might notice that I already added the option of using native tools for make staticrequired if they exist. So much faster.

[dennisameling]

So what about I look into adding the arm64 version of drud/golang-build-container towards the end of this month (quite packed with work currently)? Just did a quick check and the Golang Docker image is available for arm64, so that's a great start. That'd be the fastest path forward for now, I guess.

[rfay]

It would be awesome!

[rfay]

Oh, probably a better technique: Try to use the upstream Golang docker image and abandon our own. Many of the extra tools installed in ours aren't needed any more anyway. That way we'd lose one more thing to maintain! Much easier.

[rfay]

For build support, https://github.com/drud/ddev/pull/2549 should get us there. I have trouble with the build on Linux right now, but that should be solvable.

[rfay]

The build works now in https://github.com/drud/ddev/pull/2549 so we should be able to go forward with tests. I'll probably try with the raspberry-pi

[rfay]

I ran make test with GOTEST_SHORT and had only these failures:

--- FAIL: TestDdevXdebugEnabled (60.00s)
--- FAIL: TestDdevAllDatabases (289.91s)
--- FAIL: TestGetAppsEmpty (0.03s)
--- FAIL: TestRouterNotRunning (0.01s)
--- FAIL: TestListWithoutDir (0.17s)
--- FAIL: TestRouterConfigOverride (286.50s)
--- FAIL: TestSSHAuth (143.38s)
--- FAIL: TestCmdStop (177.07s)
--- FAIL: TestCmdAuthSSH (58.13s)

[rfay]

A piece of good news: PHP5.6 is now available in arm64 on deb.sury.org repo. Updated in https://github.com/drud/ddev-images/releases/tag/v0.5.3

However, php8.0 is not yet in arm64.

[rfay]

FYI, php8.0 is now in arm64 builds as of ddev v1.16.0-alpha7.

However, it has no xdebug, so https://github.com/drud/ddev/commit/c3ef28043df1ce9fba1b58b0e0cec950fcdad954 turned off xdebug testing in 8.0, the reminder issue is https://github.com/drud/ddev/issues/2566