Singularity: How to build a singularity image from a local docker image (not a registry image)?

[bbarker]

Version of Singularity:

2.5.1

Expected behavior

Documented way to create a singularity image from a local docker image (not an image on a registry)

Looked at documentation here.

Actual behavior

brandon@beb82dell0:~/Downloads$ docker images | grep testing8
nix_ubuntu                       testing8                         bea3190832c4        25 hours ago        10.1GB
brandon@beb82dell0:~/Downloads$ singularity build nix_ubuntu_testing8.img docker://nix_ubuntu:testing8
Docker image path: index.docker.io/library/nix_ubuntu:testing8
ERROR UNAUTHORIZED: authentication required
ERROR Check existence, naming, and permissions
Cleaning up...

Steps to reproduce behavior

See above

[GodloveD]

Hi @bbarker. I'm going to copy and paste the description from @dctrud here.

# Start a docker registry
$ docker run -d -p 5000:5000 --restart=always --name registry registry:2

# Push local docker container to it
$ docker tag alpine localhost:5000/alpine
$ docker push localhost:5000/alpine

# Create def file for singularity like this..
# (add your modifications)
Bootstrap: docker
Registry: http://localhost:5000
Namespace:
From: alpine:latest

# Build singularity container
$ sudo SINGULARITY_NOHTTPS=1 singularity build alpine.simg def

You're right, this should really be in the Singularity docs. Do you feel like submitting a PR to https://github.com/singularityware/singularityware.github.io/tree/docs/2.5? If not, I can try to get to it when I have a minute. Thanks!

[holgerbrandl]

@GodloveD @psteinb It seems that this approach requires root access on the host. Without being root it (2.4-dist) just says:

> SINGULARITY_NOHTTPS=1 singularity build foo.simg def
Using container recipe deffile: def
ERROR: You must be the root user to build from a definition file

Is there another workaround for this step which does not require sudo?

[GodloveD]

I'm going to ping @dctrud and @vsoch on this. I know you can do something like this without sudo:

$ singularity build my_image docker://me/my_image

But don't know if you can supply a custom URI (localhost) with that syntax.

[dtrudg]

It should work like this...

SINGULARITY_NOHTTPS=1 singularity build alpine.simg docker://localhost:5000/alpine:latest

[holgerbrandl]

Thanks @dctrud for the advice. Works great for me.

[bilke]

Just a reference to #2255 that this does not work with 3.0 at the moment.

[jscook2345]

Closing since this is now #2255

[dahlo]

Just a note to anyone else trying to do this, you can get this to work by specifying docker-daemon:// instead of just docker:// in your singularity build command. No need to start your own docker registry and pushing it there. As long as you have the docker image on your computer it will use the cached layers in your docker daemon directly.

Using OPs command:
singularity build nix_ubuntu_testing8.img docker-daemon://nix_ubuntu:testing8

It would be nice if it could be mentioned in the Overview list of targets in https://sylabs.io/guides/3.5/user-guide/build_a_container.html

[dtrudg]

The original question here was on a 2.x version of Singularity - for the benefit of anyone using 2.6 still, docker-daemon is not available there. It is a 3.x feature.

@dahlo - yes we could add docker-daemon to the page you link. There are other (less commonly usd) sources not mentioned in that quick list. docker-archive and oci archives are supported too.

[efratushava]

@GodloveD and everyone,
I also encountered this problem.

I'm using singularity 2.6.0-dist, without root permissions (I don't have docker installed).

How can I run a local docker image?

Running: singularity run docker://efrat/Dockerfile (My dockerfile is in the efrat directory)
Gives out: ERROR : Authentication error, exiting.

What can I do?

[dtrudg]

@efratushava - In 2.x you have to build the docker image with docker somewhere, push it to a registry (either DockerHub, something else, or a local registry), than have Singularity pull from that registry, per the comment above. It is not possible to just build or run from Dockerfile

If you need to keep things local, you will need to run a local docker registry somewhere you do have docker, per the comment above:

https://github.com/sylabs/singularity/issues/1537#issuecomment-388642244

[durai23]

Just a note to anyone else trying to do this, you can get this to work by specifying docker-daemon:// instead of just docker:// in your singularity build command. No need to start your own docker registry and pushing it there. As long as you have the docker image on your computer it will use the cached layers in your docker daemon directly.

Using OPs command:
singularity build nix_ubuntu_testing8.img docker-daemon://nix_ubuntu:testing8

It would be nice if it could be mentioned in the Overview list of targets in https://sylabs.io/guides/3.5/user-guide/build_a_container.html

@dahlo while this works, I have to be root for it to work. If I run as regular user I get:

INFO: Starting build... FATAL: While performing build: conveyor failed to get: Error loading image from docker engine: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.22/images/get?names=alpine%3Alatest": dial unix /var/run/docker.sock: connect: permission denied

This is singularity version 3.5.2

[dahlo]

@durai23 It looks like your user is not a member of the docker group. If you add it as a member you should be able to do it without sudo.

Cheers