podman play kube not utilizing existing or locally built images

Created on 12 Feb 2020 · 1Comment · Source: containers/podman

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)
/kind bug

Description
podman play kube does not seem to be utilizing local images. I am going back to a bug I found in a project of mine so this deployment has worked in previous versions of podman.

Steps to reproduce the issue:

podman play k8s-localtest-deployment.yaml

Describe the results you received:
It failed to get the image needed to deploy the pod.

Describe the results you expected:
As far as I understand k8s and podman's play kube, it should first check to see if a container image is present before attempting to pull one remotely (this behavior can be modified with imagePullPolicy). It doesn't appear to detect and/or utilize the local images I have built and fails pulling from public registries as the image doesn't exist there.

It keeps trying to pull nginx too which is confusing as it is already as present as witnessed in the output I post below.

Additional Things I've tried:

Change image in yml from mswebapps:latest to localhost/mswebapps:latest. Result: Fails to pull images from localhost on 443. It seems to be expecting my localhost to be operating a registry.
Attempt to tag localhost/mswebapps:latest as mswebapps:latest. Result: Podman seems to always prepend localhost/

The image was built with buildah

A stripped snippet from the k8s pod yaml

containers:
  - name: nginx
    image: nginx:latest
  - name: mswebapps
    image: mswebapps:latest
  - name: mslegacyapi
    image: mslegacy:latest
  - name: msmanagementapi
    image: msmanagement:latest

snippet from podman images

localhost/mswebapps          latest   40fa3d9d6ebe   37 minutes ago   1.19 GB
localhost/msmanagement       latest   1451cf1f64fc   37 minutes ago   1.15 GB
localhost/mslegacy           latest   fab0ce3ac08d   38 minutes ago   1.17 GB

podman play kube

Trying to pull docker.io/library/mswebapps:latest...
  denied: requested access to the resource is denied
Trying to pull registry.fedoraproject.org/mswebapps:latest...
  manifest unknown: manifest unknown
Trying to pull registry.access.redhat.com/mswebapps:latest...
  name unknown: Repo not found
Trying to pull registry.centos.org/mswebapps:latest...
  manifest unknown: manifest unknown
Trying to pull quay.io/mswebapps:latest...
  error parsing HTTP 404 response body: invalid character '<' looking for beginning of value: "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 3.2 Final//EN\">\n<title>404 Not Found</title>\n<h1>Not Found</h1>\n<p>The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.</p>\n"
Error: unable to pull mswebapps:latest: 5 errors occurred:
    * Error initializing source docker://mswebapps:latest: Error reading manifest latest in docker.io/library/mswebapps: errors:
denied: requested access to the resource is denied
unauthorized: authentication required

    * Error initializing source docker://registry.fedoraproject.org/mswebapps:latest: Error reading manifest latest in registry.fedoraproject.org/mswebapps: manifest unknown: manifest unknown
    * Error initializing source docker://registry.access.redhat.com/mswebapps:latest: Error reading manifest latest in registry.access.redhat.com/mswebapps: name unknown: Repo not found
    * Error initializing source docker://registry.centos.org/mswebapps:latest: Error reading manifest latest in registry.centos.org/mswebapps: manifest unknown: manifest unknown
    * Error initializing source docker://quay.io/mswebapps:latest: Error reading manifest latest in quay.io/mswebapps: error parsing HTTP 404 response body: invalid character '<' looking for beginning of value: "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 3.2 Final//EN\">\n<title>404 Not Found</title>\n<h1>Not Found</h1>\n<p>The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.</p>\n"

Output of podman version:

Version:            1.8.0
RemoteAPI Version:  1
Go Version:         go1.13.6
OS/Arch:            linux/amd64

Output of podman info --debug:

debug:
  compiler: gc
  git commit: ""
  go version: go1.13.6
  podman version: 1.8.0
host:
  BuildahVersion: 1.13.1
  CgroupVersion: v2
  Conmon:
    package: conmon-2.0.10-2.fc31.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.10, commit: 6b526d9888abb86b9e7de7dfdeec0da98ad32ee0'
  Distribution:
    distribution: fedora
    version: "31"
  IDMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  MemFree: 42433658880
  MemTotal: 67224588288
  OCIRuntime:
    name: crun
    package: crun-0.12.1-1.fc31.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 0.12.1
      commit: df5f2b2369b3d9f36d175e1183b26e5cee55dd0a
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
  SwapFree: 34359734272
  SwapTotal: 34359734272
  arch: amd64
  cpus: 8
  eventlogger: journald
  hostname: DFP-Xavier-Loose
  kernel: 5.4.17-200.fc31.x86_64
  os: linux
  rootless: true
  slirp4netns:
    Executable: /usr/bin/slirp4netns
    Package: slirp4netns-0.4.0-20.1.dev.gitbbd6f25.fc31.x86_64
    Version: |-
      slirp4netns version 0.4.0-beta.3+dev
      commit: bbd6f25c70d5db2a1cd3bfb0416a8db99a75ed7e
  uptime: 8m 17.44s
registries:
  search:
  - docker.io
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - registry.centos.org
  - quay.io
store:
  ConfigFile: /home/xav/.config/containers/storage.conf
  ContainerStore:
    number: 7
  GraphDriverName: overlay
  GraphOptions:
    overlay.mount_program:
      Executable: /usr/bin/fuse-overlayfs
      Package: fuse-overlayfs-0.7.5-2.fc31.x86_64
      Version: |-
        fusermount3 version: 3.6.2
        fuse-overlayfs: version 0.7.5
        FUSE library version 3.6.2
        using FUSE kernel interface version 7.29
  GraphRoot: /home/xav/.local/share/containers/storage
  GraphStatus:
    Backing Filesystem: btrfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  ImageStore:
    number: 16
  RunRoot: /run/user/1000
  VolumePath: /home/xav/.local/share/containers/storage/volumes

Package info (e.g. output of rpm -q podman or apt list podman):

podman-1.8.0-2.fc31.x86_64

Additional environment details (AWS, VirtualBox, physical, etc.):
physical

kinbug

Source

xavloose

Most helpful comment

After looking through the source I learned it had to do with my configuration via the following comment in libpod/pods.go:

// In kube, if the image is tagged with latest, it should always pull

This was a misunderstanding by myself of how kube works. Removing the latest tag in my pod yaml fixed the issue.

xavloose on 12 Feb 2020

👍4

>All comments

After looking through the source I learned it had to do with my configuration via the following comment in libpod/pods.go:

// In kube, if the image is tagged with latest, it should always pull

This was a misunderstanding by myself of how kube works. Removing the latest tag in my pod yaml fixed the issue.

xavloose on 12 Feb 2020

👍4

Was this page helpful?

0 / 5 - 0 ratings

Related issues

Error pulling image , Error committing the finished image: error adding layer with blob ...

juansuerogit · 5Comments

`podman play kube` (rootless) does not set `command` and `args` as specfied at the k8s specs

cruwe · 3Comments

Unable to find user $USER: no matching entries in passwd file

runlevel5 · 4Comments

/usr/bin/podman version returns Error: default OCI runtime "crun" not found: invalid argument

raukadah · 4Comments

COPY misbehaves in the presence of a non-empty .dockerignore file

undecaf · 3Comments