Buildx: How to manage target-specific build options?

[Julio-Guerra]

Hello,

I am facing a common situation when compiling for different architectures: having specific compilation flags (like hardware or software floats, etc.).

The Dockerfile is strictly the same other than these extra flags specific to the target platform. So I was wondering what would be the best way to use have a single docker buildx build --platform=linux/amd64,linux/arm64 but with many platform-specific args?

I was thinking of adding an ARG CFLAGS but I would require to make two separate calls to docker buildx, one for each platform.

Thanks a lot

[tonistiigi]

bit hacky but you can do:

FROM alpine AS base

FROM base AS base-amd64
ARG FOO_AMD64
ARG FOO=$FOO_AMD64

FROM base AS base-386
ARG FOO_386
ARG FOO=$FOO_386

FROM base-$TARGETARCH
RUN env

# docker buildx build --progress=plain --build-arg FOO_AMD64=8bytes --build-arg FOO_386=4bytes --platform=linux/amd64,linux/386 .
WARN[0000] No output specified for docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load
#2 [internal] load .dockerignore
#2 transferring context: 2B done
#2 DONE 0.0s

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 200B done
#1 DONE 0.0s

#4 [linux/386 internal] load metadata for docker.io/library/alpine:latest
#4 DONE 0.5s

#3 [linux/amd64 internal] load metadata for docker.io/library/alpine:latest
#3 DONE 0.5s

#5 [linux/386 base 1/1] FROM docker.io/library/alpine@sha256:72c42ed48c3a2d...
#5 resolve docker.io/library/alpine@sha256:72c42ed48c3a2db31b7dafe17d275b634664a708d901ec9fd57b1529280f01fb done
#5 CACHED

#7 [linux/amd64 base 1/1] FROM docker.io/library/alpine@sha256:72c42ed48c3a...
#7 resolve docker.io/library/alpine@sha256:72c42ed48c3a2db31b7dafe17d275b634664a708d901ec9fd57b1529280f01fb done
#7 CACHED

#6 [linux/386 stage-3 1/1] RUN env
#6 0.143 FOO_386=4bytes
#6 0.143 SHLVL=1
#6 0.143 HOME=/root
#6 0.143 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
#6 0.143 FOO=4bytes
#6 0.143 PWD=/
#6 DONE 0.2s

#8 [linux/amd64 stage-3 1/1] RUN env
#8 0.155 FOO_AMD64=8bytes
#8 0.155 SHLVL=1
#8 0.155 HOME=/root
#8 0.155 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
#8 0.155 FOO=8bytes
#8 0.155 PWD=/
#8 DONE 0.2s

[tiborvass]

I think the easiest solution is to use a wrapper script. For example, in this cross-compilation helper script we set up the C compiler with the CC variable basing it off of GOARCH which itself is based on TARGETARCH: https://github.com/tonistiigi/xx/blob/master/golang/wrapper.sh#L62-L82

[Johannestegner]

_Disclaimer: I haven't actually tried this myself. You might also already have tried it, else, it's probably worth a shot!_

If you define the arch specific arguments as for example: CC_arm64 and then, when you define your env variable use something like the following:

ARG CC_arm64
ARG TARGETARCH
ENV CC="CC_${TARGETARCH}"

You should be able to at the least not have to run some extra scripts inside the build script.
This will of course bloat the Dockerfile with a few extra ARG's... but yeah....

In case it does not work (due to dockers env substitution not being as "advanced" as the sh one, you could probably do it inside a RUN command:

RUN CC="CC_${TARGETARCH}" \
  && make

[Julio-Guerra]

Solved using @tonistiigi technique of having a per $TARGETARCH base that sets specific environment variable. I couldn't set them in the Makefile of third-party libraries so it necessarily had to come from the build environment.

Complete example:

FROM gcc:9 AS base

FROM base AS base-amd64
ENV TARGET_CMAKE_C_FLAGS "-Wa,-mrelax-relocations=no"

FROM base AS base-arm64
ENV TARGET_CMAKE_C_FLAGS ""

ARG TARGETARCH
FROM base-$TARGETARCH AS libcxx-dev
RUN env