Singularity: Singularity cannot run a local Docker image directly

[alalazo]

Version of Singularity:

$ singularity version
3.5.0-rc.2

Expected behavior

If I have a local docker image I expect to be able to run it directly using the docker-daemon bootstrap agent:

$ docker images alpine-mod
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
alpine-mod          latest              4f0df2bb4d29        10 minutes ago      5.55MB

$ singularity run docker-daemon://alpine-mod:latest <command>

Actual behavior

Trying to run the image returns an error:

$ singularity run docker-daemon://alpine-mod:latest cat /etc/os-release
FATAL:   Unable to handle docker-daemon://alpine-mod:latest uri: failed to get SHA of docker-daemon://alpine-mod:latest: unable to parse image name docker-daemon://alpine-mod:latest: invalid reference format

It is possible though to convert the image first and run the sif file later:

$ singularity build alpine-mod.sif docker-daemon://alpine-mod:latest
INFO:    Starting build...
Getting image source signatures
Copying blob 77cae8ab23bf done
Copying blob d441ecfcd056 done
Copying config 6957fd9134 done
Writing manifest to image destination
Storing signatures
2019/11/13 15:00:23  info unpack layer: sha256:eb7c47c7f0fd0054242f35366d166e6b041dfb0b89e5f93a82ad3a3206222502
2019/11/13 15:00:23  info unpack layer: sha256:0240d3f369abb5ab81827f8cd4b9107bccc7a0969ed051b7eea95050f3acbf35
INFO:    Creating SIF file...
INFO:    Build complete: alpine-mod.sif

$ singularity run alpine-mod.sif cat /etc/os-release
NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.10.3
PRETTY_NAME="Alpine Linux v3.10"
HOME_URL="https://alpinelinux.org/"
BUG_REPORT_URL="https://bugs.alpinelinux.org/"

Steps to reproduce this behavior

$ echo -e "FROM alpine\nRUN touch /opt/delete.me" | docker build -t alpine-mod -
Sending build context to Docker daemon  2.048kB
Step 1/2 : FROM alpine
 ---> 965ea09ff2eb
Step 2/2 : RUN touch /opt/delete.me
 ---> Using cache
 ---> 4f0df2bb4d29
Successfully built 4f0df2bb4d29
Successfully tagged alpine-mod:latest

$ singularity run docker-daemon://alpine-mod:latest ls /opt
FATAL:   Unable to handle docker-daemon://alpine-mod:latest uri: failed to get SHA of docker-daemon://alpine-mod:latest: unable to parse image name docker-daemon://alpine-mod:latest: invalid reference format

$ singularity build alpine-mod.sif docker-daemon://alpine-mod:latest
INFO:    Starting build...
Getting image source signatures
Copying blob 77cae8ab23bf done
Copying blob d441ecfcd056 done
Copying config 6957fd9134 done
Writing manifest to image destination
Storing signatures
2019/11/13 15:04:54  info unpack layer: sha256:eb7c47c7f0fd0054242f35366d166e6b041dfb0b89e5f93a82ad3a3206222502
2019/11/13 15:04:54  info unpack layer: sha256:0240d3f369abb5ab81827f8cd4b9107bccc7a0969ed051b7eea95050f3acbf35
INFO:    Creating SIF file...
INFO:    Build complete: alpine-mod.sif

$ singularity run alpine-mod.sif ls /opt
delete.me

What OS/distro are you running

$ cat /etc/os-release 
NAME="Ubuntu"
VERSION="18.04.3 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.3 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

How did you install Singularity

Manually from the official rc tarball.

[dtrudg]

I'm afraid this is indeed a bug. The // are not being handled properly by the run flow in the code.

Please try singularity run docker-daemon:alpine-mod:latest as a work around at this time.

[dtrudg]

Hi @alalazo

Looking back at this, we use the containers/image package for transport handling. In containers/ image the correct format for docker-archive is docker-archive:alpine:latest (without the //) The value with the // only works in build as a side-effect of handling the // on docker sources in def file creation. (docker, oras etc. sources do correctly include a //)

The documentation here: https://sylabs.io/guides/3.5/user-guide/appendix.html#docker-daemon-docker-archive-bootstrap-agents is incorrect and needs to be fixed.

It's a bit confusing, but I don't really want to add things to handle the // in Singularity itself, as we are then drifting from the correct URIs for the library, which is also used in Skopeo and other container tools.

[alalazo]

@dctrud I'll submit a change to the docs tomorrow.