Cockroach: Run CockroachDB on arm64

[tirithen]

What would it take to provide binaries for arm64 architecture for CockroachDB? I have seen issues around building it (e.g. https://github.com/cockroachdb/cockroach/issues/18319) and this weekend I have tried my self learning about cross compilation but with my limited knowledge I was unable to find any pre configured tools (like dockcross and so on) that could provide me with a cross compilation environment easy enough for me to grasp.

I want to try CockroachDB on Orange Pi PC2 (similar specs as Raspberry Pi 3) but I have a hard time thinking CockroachDB can be built on that board.

If I can contribute in getting it built and distributed for arm64 I would try to but after spending some hours this weekend trying to figure out how cross compilation works I realized that it's a bit overwhelming for me at this point. Is there some good guidance on how to get going with compiling this for arm64 or even any ongoing plans to distribute this as a binary. It looks like some people has managed to compile this already.

[tbg]

cc @nvanbenschoten, who's been the driving force behind arm builds.

[sercand]

@tirithen I have built before. You can use https://hub.docker.com/r/sercand/cockroach-arm/ docker image.

[tirithen]

Thanks! I'll try that one out. :)

[fire]

@nvanbenschoten Can you provide arm64 builder docker images?

[nvanbenschoten]

@fire we don't currently have an official arm64 builder docker image or an official arm64 cockroach docker image. I'm going to take a look at how difficult these would be to create.

[nvanbenschoten]

@fire I just opened a PR that sets up an aarch64 cross-compilation toolchain in our docker builder image. This will allow us/you to build arm64 release binaries on x86 machines by running

build/builder.sh make build TYPE=release-aarch64-linux

We can put this arm64 binary in an arm64 Docker image pretty easily too, although I'm not sure if that's what you were originally asking for.

Do note that these will not be "official" binaries, as we still don't officially support ARM. What this means in practice is that we don't perform enough internal testing and qualification on ARM to feel comfortable stamping it as fully supported. That said, I'm not aware of any issues at the moment and our automated testing suite passes cleanly on the platform.

[fire]

Cheers, that's what I'm looking for.

[tirithen]

The build script that you provided works well to build and the binary is possible to run on my arm64 machines. However when I try to run them from a docker image (FROM scratch or alpine:3.7) I get the following error:

standard_init_linux.go:195: exec user process caused "no such file or directory"

Is this maybe related to the build needing CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build to become fully stand alone binary? Also I guess CGO_ENABLED=0 will not work since there are C code in the cockroach project?

This might be considered off-topic but I still figured that someone more experienced here would have an answer.

[tirithen]

Okay, when using FROM debian everything works fine, I guess alpine is just missing something that is needed by cockroachdb, I'll just stick to debian for now.

[nvanbenschoten]

@tirithen I followed the instructions from here and did not run into the issue you saw.

To do this, I ran the build/builder.sh make build TYPE=release-aarch64-linux on an x86 machine (because https://github.com/cockroachdb/cockroach/pull/24442 is proving troublesome). I then transferred the resulting binary to an arm64 machine and followed the second and third steps from here.

For ease of use, I just pushed the resulting image to Dockerhub under the cockroachdb/cockroach-unstable-arm64v8 repository. This image is an arm64 version of the v2.0 release we announced yesterday.

[tirithen]

Thanks for the info, and image. Trying to configure this into my swarm mode cluster now.

[jrote1]

@sercand @nathanejohnson hi do either of you have the Dockerfiles you used to create these images? Thanks in advance

[SvenDowideit]

@nvanbenschoten it looks like the build image on Docker hub hasn't been updated in a long time

is there a technical problem with cockroachdb on rpi / arm64? or is it just not something that has a commercial priority?

(I'm basically wondering if its worth trying to get a build working, or if its not going to work for other reasons)

(I'd have tried already, but my build server's in pieces atm :) )

see https://forum.cockroachlabs.com/t/failing-make-build-of-20-1-0-source-on-ubuntu-20-04-arm64/3701

[nvanbenschoten]

is there a technical problem with cockroachdb on rpi / arm64? or is it just not something that has a commercial priority?

Running cockroach on arm64 just hasn't been a large priority so we don't publish arm64 Docker images, but we still do maintain a cross-compilation toolchain that targets arm64. On your build machine, you should be able to run build/builder.sh mkrelease arm64-linux. The only dependency there is Docker.

[SvenDowideit]

Excellent news - thanks!

On Wed, May 27, 2020 at 4:41 AM Nathan VanBenschoten <
[email protected]> wrote:

is there a technical problem with cockroachdb on rpi / arm64? or is it
just not something that has a commercial priority?

Running cockroach on arm64 just hasn't been a large priority so we don't
publish arm64 Docker images, but we still do maintain a cross-compilation
toolchain that targets arm64. On your build machine, you should be able to
run build/builder.sh mkrelease arm64-linux. The only dependency there is
Docker.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/cockroachdb/cockroach/issues/20433#issuecomment-634205077,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAG6THB6MJZYJBCJO6WPFTRTQEMBANCNFSM4EGPVRBQ
.

--

Professional Wiki Innovation and Support
Sven Dowideit - http://DistributedINFORMATION.com
A WikiRing Partner http://wikiring.com