Cli: Image Manifest Override

[EmbeddedAndroid]

Problem description

Using the image manifest schema I can describe multiple architecture images under a single namespace and tag.

For my base images, I use tools like debootstrap, mkimage-alpine, etc to create compressed root file systems imported as docker layers. These tools allow the user to specify a target architecture, to allow the users to create ABI compatible root file systems on a single host. As an example, arm64 targets are ABI compatible with armhf, I can also produce armhf tar balls on my arm64 host without issue.

After I've imported the root file systems, I then create a manifest to create a single multi arch image from these outputs.

image: linarotechnologies/alpine:3.6
manifests:
  -
    image: linarotechnologies/alpine:3.6-arm64
    platform:
      architecture: arm64
      os: linux
  -
    image: linarotechnologies/alpine:3.6-armhf
    platform:
      architecture: arm
      os: linux
  -
    image: alpine:3.6
    platform:
      architecture: amd64
      features:
        - sse
      os: linux

Everything works beautifully, on x86, arm64, or armhf hosts! However, the issue I'm facing now is when building container based on this multi-arch base images. Take for example the following Dockerfile statement:

FROM linarotechnologies/alpine:3.6

When building this Dockerfile, the image which is pulled matches the docker host's architecture. What happens if a user wishes to build an ABI compatible image using another architecture? Some examples:

amd64 host wishes to build an i386 image
arm64 host wishes to build an arm image
mips64 host wishes to build an mips image

There may be more architectures with ABI compatibility but this gives you a basic idea of the problem.

Project version(s) affected

Latest Docker Version (Tested with v1.13)

Suggestions for a fix

I would suggest docker implements a simple --arch flag, which would force the specified architecture to be used when resolving the image from the manifest.

docker build -t myimage --arch=armhf -f Dockerfile .
docker run -it --arch=armhf myimage

[thaJeztah]

ping @justincormack I think I saw you opened an issue about cross-compiled images, but forgot where I saw it

[ghost]

I do not see a way to build docker images for ARM on x86_64 hosts (with travis specifically) if the base image has a manifest. I wanted to build an image based on _/python. Since this always pulls the base image for linux/amd64 I tried to use arm32v7/python:3, which failed with no matching manifest for linux/amd64 in the manifest list entries.

Is there some workaround that I overlooked or is multiarch building with manifests simply currently impossible?

[EmbeddedAndroid]

@eqrx I have found no "good" workaround other than building natively. It's faster but very inconvenient.

[agronholm]

There is the --platform switch which is available when you enable experimental features, but it only works for OS switching, contrary to what its documentation says. The code is supposed to allow you to switch architecture too but at the moment it's been hard coded to reject all architectures as invalid.

[vielmetti]

This question came up again in Slack at

https://dockercommunity.slack.com/archives/C2293P89Y/p1517610051000453

[daniel-kun]

It was me asking the question.
My use-case is building Raspberry Pi images on Travis CI (or my dev box, or any other CI) from multiarch-manifests. I would've used it for https://github.com/daniel-kun/kube-alive - now I resort to specify arch-specific images to FROM.

[deitch]

Funny, I was just discussing that with someone on the Docker Community Slack in a private message. As far as I know, there is no way now to docker run --arch=arm64 alpine:3.6 which would, if a manifest exists for alpine:3.6, override using the engine's running arch for arm64.

Which component decides which arch to pull from the multi-arch manifest? Obviously the engine does the actual "pull manifest and find a particular arch and pull that image", but the "particular arch" to find is decided by the _engine_? Or does the CLI query the engine for the arch and then tell it "pick this arch"? I assume the former, but asking.

If someone can point me to the code in the engine and then the CLI, happy to do a PR for an option to override.

[suihkulokki]

An ugly workaround is use docker manifest to pick up the sha of the target image you want to use:

docker manifest list debian:buster
...
         "digest": "sha256:971e2f276da3f487ab81506a2fa9f7fda93ad061637f8e7277ffa1ac9400ccf3",
         "platform": {
            "architecture": "arm",
            "os": "linux",   
            "variant": "v7"  
         }
...
docker run  --rm  -it debian@sha256:971e2f276da3f487ab81506a2fa9f7fda93ad061637f8e7277ffa1ac9400ccf3 /bin/bash
...
dpkg --version
Debian 'dpkg' package management program version 1.19.0.5 (armhf).

[vielmetti]

If you start with the manifest file from

docker manifest inspect debian:buster > buster.json

then this bit of jq will select out the target digest for the appropriate platform architecture

cat buster.json | jq '.manifests[] | select(.platform.architecture == "arm64") | .digest'

[SamiSousa]

I believe this has been addressed at least partly by moby/moby#37350

[agronholm]

Does anyone know when --platform=X will be available? There wasn't a Docker CE release for June, so maybe in July's release?

[ghost]

I would like this, my use case is to test images for RPI on my Windows PC. This works because the current Docker version actually runs/emulates different architectures.
Note this recent post: https://engineering.docker.com/2019/04/multi-arch-images/
This goes into the emulation support, but even in this recent post the manifest needs to be inspected, and the hashes used, to actually run the different architecture. :)