Buildah: Document how to use buildah with local docker image

[stiller-leser]

Hi,

I've been trying to get buildah bud . Dockerfile to run with a local docker image (suse/sles12sp12) on a Suse system.

I have tried adding docker:// before the image in the FROM clause as well as docker-daemon: but none of these seems to work. Buildah still returns error building: error creating build container: no such image "docker://suse/sles12sp2:1.0.2" in registry: image not known.

I was looking into installing skopeo to try and see if that helps, however I gave up since ostree isn't really available on Suse.

I'm sure that I am missing something, but maybe that would be something worth documenting.

[rhatdan]

So the image is installed in the docker daemons storage?

I would think you can use docker-daemon://suse/sles12sp2:1.0.2

If you use docker:// that means you are using the "docker" protocol to pull from a container registry. docker-daemon indicates the storage is in a docker-daemon. Not sure if this was ever tried before though.

skopeo should be able to build without ostree?

[stiller-leser]

Here's what I tried - unfortunately to no avail:

docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
suse/sles12sp2      1.0.2               0cfb79e33542        8 minutes ago       134MB
suse/sles12sp2      latest              0cfb79e33542        8 minutes ago       134MB
centos              7                   2d194b392dd1        3 days ago          195MB
alpine              latest              3fd9065eaf02        8 weeks ago         4.14MB

buildah from docker-daemon://suse/sles12sp2:1.0.2
no such image "docker-daemon://suse/sles12sp2:1.0.2" in registry: image not known



md5-07631c773b42d3b5f75f2599f2a39637



buildah from centos
no such image "centos" in registry: image not known



md5-07631c773b42d3b5f75f2599f2a39637



buildah version
Version:       0.15
Go Version:    go1.9.4
Image Spec:    1.0.0
Runtime Spec:  1.0.0
Git Commit:    1211481
Built:         Thu Mar  8 14:56:23 2018
OS/Arch:       linux/amd64



md5-07631c773b42d3b5f75f2599f2a39637



lsb_release -a
Distributor ID: SUSE
Description:    SUSE Linux Enterprise Server 12 SP3
Release:    12.3



md5-daea5077bfb892bfb59b268f709d6dc7



cd $GOPATH/src/github.com/projectatomic/skopeo && make binary-local
go build "-buildmode=pie" -ldflags "-X main.gitCommit=fd0c3d7f08ceb4369695854e6e50fdf9ac09f87a" -gcflags "" -tags "  " -o skopeo ./cmd/skopeo
# pkg-config --cflags ostree-1 ostree-1 ostree-1 ostree-1 ostree-1 ostree-1
Package ostree-1 was not found in the pkg-config search path.
Perhaps you should add the directory containing `ostree-1.pc'
to the PKG_CONFIG_PATH environment variable
No package 'ostree-1' found
Package ostree-1 was not found in the pkg-config search path.
Perhaps you should add the directory containing `ostree-1.pc'
to the PKG_CONFIG_PATH environment variable
No package 'ostree-1' found
Package ostree-1 was not found in the pkg-config search path.
Perhaps you should add the directory containing `ostree-1.pc'
to the PKG_CONFIG_PATH environment variable
No package 'ostree-1' found
Package ostree-1 was not found in the pkg-config search path.
Perhaps you should add the directory containing `ostree-1.pc'
to the PKG_CONFIG_PATH environment variable
No package 'ostree-1' found
Package ostree-1 was not found in the pkg-config search path.
Perhaps you should add the directory containing `ostree-1.pc'
to the PKG_CONFIG_PATH environment variable
No package 'ostree-1' found
Package ostree-1 was not found in the pkg-config search path.
Perhaps you should add the directory containing `ostree-1.pc'
to the PKG_CONFIG_PATH environment variable
No package 'ostree-1' found
pkg-config: exit status 1
Makefile:78: recipe for target 'binary-local' failed
make: *** [binary-local] Error 2

Let me know if I can provide any additional information. This is all I could come up with as useful.

[umohnani8]

Looks like buildah from doesn't support the various transports. Will add in the functionality.
@stiller-leser we support pulling from registries, if you want to be able to do buildha from centos then edit your /etc/containers/registries.conf file and add the registries you want buildah to try and pull an image from.

[stiller-leser]

Hi @umohnani8,

thanks for the feedback. For what it is worth I do not even have a /etc/containers/registries.conf. I created one based on the one found in /tests. With it, I had the following output:

buildah --debug from centos7
DEBU[0000] overlay test mount with multiple lowers succeeded 
DEBU[0000] backingFs=btrfs, projectQuotaSupported=false, useNativeDiff=true 
INFO[0000] [graphdriver] using prior storage driver: overlay 
DEBU[0000] parsed reference to refname into "[overlay@/var/lib/containers/storage+/var/run/containers/storage]docker.io/centos7:latest" 
DEBU[0000] parsed reference to refname into "[overlay@/var/lib/containers/storage+/var/run/containers/storage]docker.io/centos7:latest" 
DEBU[0000] reference "[overlay@/var/lib/containers/storage+/var/run/containers/storage]docker.io/centos7:latest" does not resolve to an image ID 
DEBU[0000] parsed reference to refname into "[overlay@/var/lib/containers/storage+/var/run/containers/storage]docker.io/centos7:latest" 
DEBU[0000] parsed reference to refname into "[overlay@/var/lib/containers/storage+/var/run/containers/storage]docker.io/centos7:latest" 
DEBU[0000] Using registries.d directory /etc/containers/registries.d for sigstore configuration 
DEBU[0000]  No signature storage configuration found for docker.io/centos7:latest 
DEBU[0000] Looking for TLS certificates and private keys in /etc/docker/certs.d/docker.io
DEBU[0000] error pulling image "docker.io/centos7": error obtaining default signature policy: open /etc/containers/policy.json: no such file or directory 
DEBU[0000] error pulling and reading image "docker.io/centos7": error obtaining default signature policy: open /etc/containers/policy.json: no such file or directory 
DEBU[0000] parsed reference to refname into "[overlay@/var/lib/containers/storage+/var/run/containers/storage]registry.fedoraproject.org/centos7:latest" 
DEBU[0000] parsed reference to refname into "[overlay@/var/lib/containers/storage+/var/run/containers/storage]registry.fedoraproject.org/centos7:latest" 
DEBU[0000] reference "[overlay@/var/lib/containers/storage+/var/run/containers/storage]registry.fedoraproject.org/centos7:latest" does not resolve to an image ID 
DEBU[0000] parsed reference to refname into "[overlay@/var/lib/containers/storage+/var/run/containers/storage]registry.fedoraproject.org/centos7:latest" 
DEBU[0000] parsed reference to refname into "[overlay@/var/lib/containers/storage+/var/run/containers/storage]registry.fedoraproject.org/centos7:latest" 
DEBU[0000] Using registries.d directory /etc/containers/registries.d for sigstore configuration 
DEBU[0000]  No signature storage configuration found for registry.fedoraproject.org/centos7:latest 
DEBU[0000] Looking for TLS certificates and private keys in /etc/docker/certs.d/registry.fedoraproject.org 
DEBU[0000] error pulling image "registry.fedoraproject.org/centos7": error obtaining default signature policy: open /etc/containers/policy.json: no such file or directory 
DEBU[0000] error pulling and reading image "registry.fedoraproject.org/centos7": error obtaining default signature policy: open /etc/containers/policy.json: no such file or directory 
DEBU[0000] parsed reference to refname into "[overlay@/var/lib/containers/storage+/var/run/containers/storage]registry.access.redhat.com/centos7:latest" 
DEBU[0000] parsed reference to refname into "[overlay@/var/lib/containers/storage+/var/run/containers/storage]registry.access.redhat.com/centos7:latest" 
DEBU[0000] reference "[overlay@/var/lib/containers/storage+/var/run/containers/storage]registry.access.redhat.com/centos7:latest" does not resolve to an image ID 
DEBU[0000] parsed reference to refname into "[overlay@/var/lib/containers/storage+/var/run/containers/storage]registry.access.redhat.com/centos7:latest" 
DEBU[0000] parsed reference to refname into "[overlay@/var/lib/containers/storage+/var/run/containers/storage]registry.access.redhat.com/centos7:latest" 
DEBU[0000] Using registries.d directory /etc/containers/registries.d for sigstore configuration 
DEBU[0000]  No signature storage configuration found for registry.access.redhat.com/centos7:latest 
DEBU[0000] Looking for TLS certificates and private keys in /etc/docker/certs.d/registry.access.redhat.com 
DEBU[0000] error pulling image "registry.access.redhat.com/centos7": error obtaining default signature policy: open /etc/containers/policy.json: no such file or directory 
DEBU[0000] error pulling and reading image "registry.access.redhat.com/centos7": error obtaining default signature policy: open /etc/containers/policy.json: no such file or directory 
ERRO[0000] no such image "centos7" in registry: image not known 

Without it, I saw the same, just shorter:

DEBU[0000] overlay test mount with multiple lowers succeeded 
DEBU[0000] backingFs=btrfs, projectQuotaSupported=false, useNativeDiff=true 
INFO[0000] [graphdriver] using prior storage driver: overlay 
DEBU[0000] unable to complete image name "centos7": open /etc/containers/registries.conf: no such file or directory 
DEBU[0000] parsed reference to refname into "[overlay@/var/lib/containers/storage+/var/run/containers/storage]docker.io/library/centos7:latest" 
DEBU[0000] parsed reference to refname into "[overlay@/var/lib/containers/storage+/var/run/containers/storage]docker.io/library/centos7:latest" 
DEBU[0000] reference "[overlay@/var/lib/containers/storage+/var/run/containers/storage]docker.io/library/centos7:latest" does not resolve to an image ID 
DEBU[0000] parsed reference to refname into "[overlay@/var/lib/containers/storage+/var/run/containers/storage]docker.io/library/centos7:latest" 
DEBU[0000] parsed reference to refname into "[overlay@/var/lib/containers/storage+/var/run/containers/storage]docker.io/library/centos7:latest" 
DEBU[0000] Using registries.d directory /etc/containers/registries.d for sigstore configuration 
DEBU[0000]  No signature storage configuration found for docker.io/library/centos7:latest 
DEBU[0000] Looking for TLS certificates and private keys in /etc/docker/certs.d/docker.io 
DEBU[0000] error pulling image "centos7": error obtaining default signature policy: open /etc/containers/policy.json: no such file or directory 
DEBU[0000] error pulling and reading image "centos7": error obtaining default signature policy: open /etc/containers/policy.json: no such file or directory 
ERRO[0000] no such image "centos7" in registry: image not known 

This I saw with no matter what image I tried. Correct me if I'm wrong, but public images should always be accessible, no? (Ignoring the special case that SLES is).

[umohnani8]

@stiller-leser you are missing the policy.json, just copy over the policy.json file in test to /etc/containers/policy.json and I believe your problem should go away. This is something we need to add in our readme, I will open an issue and fix that.
The reason you get the longer debug output with the registries.conf file is that buildah is trying all the registries in the conf file for the centos image.

[stiller-leser]

@umohnani8 that did the trick. Thank you!

[rhatdan]

Those files are packaged up in atomic-containers and skopeo-containers. Although they should be moving to skopeo-containers.

[umohnani8]

@rhatdan what if you just build buildah from source without installing skopeo-containers? Shouldn't we document this somewhere?

[rhatdan]

I think it is in install.md now.

Prior to installing Buildah, install the following packages on your Linux distro:
* make
* golang (Requires version 1.8.1 or higher.)
* bats
* btrfs-progs-devel
* bzip2
* device-mapper-devel
* git
* go-md2man
* gpgme-devel
* glib2-devel
* libassuan-devel
* libseccomp-devel
* ostree-devel
* runc (Requires version 1.0 RC4 or higher.)
* skopeo-containers

[umohnani8]

Fixed in #524