Podman: `podman auto-update` does not respect the `--override-arch=xyz` flag

[btannous]

/kind bug

Description

podman auto-update does not respect the --override-arch=xyz flag.

I'm running an updated Fedora IoT on a Raspberry Pi 3 b+, and podman only wants to pull images that are arch=arm64/v8, when many images are tagged arch=arm64. I've mitigated this via using the --override-arch=arm64 flag, which seems to work for the pull and run commands, but not auto-update.

I can get the image to update by passing in a --pull always, making sure to generate systemd with the --new flag and running systemctl restart x.service

Steps to reproduce the issue:

1. Follow example listed here: https://www.mankier.com/1/podman-auto-update
2. Replace image with something like mikenye/readsb for example.
3. podman auto-update

Note: the --override-arch=arm64 flag is setup in the systemd unit. Running podman auto-update --override-arch=arm64 isn't possible as that flag doesn't exist.

See https://hub.docker.com/r/mikenye/readsb/tags for available arch tags

See https://github.com/mikenye/docker-readsb/blob/master/buildx.sh for how the images are built.

See https://github.com/mikenye/docker-piaware/pull/33#issuecomment-643837702 for details on arm64 == arm64/v8 in this case

Describe the results you received:

error auto-updating container "<hash>": image check for "<image:tag>" failed: Error choosing image instance: no image found in manifest list for architecture arm64, variant v8, OS linux

Describe the results you expected:

An updated image running.

Output of podman version:

Version:            1.9.3
RemoteAPI Version:  1
Go Version:         go1.14.2
OS/Arch:            linux/arm64

Output of podman info --debug:

debug:
  compiler: gc
  gitCommit: ""
  goVersion: go1.14.2
  podmanVersion: 1.9.3
host:
  arch: arm64
  buildahVersion: 1.14.9
  cgroupVersion: v2
  conmon:
    package: conmon-2.0.17-1.fc32.aarch64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.17, commit: 19c059ee0c01a2fb2c425cff307fba1ad2cf7741'
  cpus: 4
  distribution:
    distribution: fedora
    version: "32"
  eventLogger: file
  hostname: adsb
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 5.6.15-300.fc32.aarch64
  memFree: 135544832
  memTotal: 984621056
  ociRuntime:
    name: crun
    package: crun-0.13-2.fc32.aarch64
    path: /usr/bin/crun
    version: |-
      crun version 0.13
      commit: e79e4de4ac16da0ce48777afb72c6241de870525
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
  os: linux
  rootless: false
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 468996096
  swapTotal: 468996096
  uptime: 47m 26.07s
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - registry.centos.org
  - docker.io
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 3
    paused: 0
    running: 3
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /var/lib/containers/storage
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "true"
  imageStore:
    number: 4
  runRoot: /var/run/containers/storage
  volumePath: /var/lib/containers/storage/volumes

Additional environment details (AWS, VirtualBox, physical, etc.):

Fedora IoT, Raspberry Pi 3b+

[btannous]

It's worth noting that it's semi confusing that the CPU architecture of this device could be either:

• aarch64
• arm64
• arm64/v8
• likely something else that I'm forgetting...

Maybe podman should try all of the above? /not sure.

Another note, I also have a few other arm devices (pi 4/pi zero, and a pinebook pro) that I use every now and then. So getting into multi-arch, lmk if there is anything I should check out =)

[mheon]

@vrothberg PTAL

[btannous]

/edited the OP to add some:

Notes

[vrothberg]

Thanks for opening the issue, @btannous! I'll take a look but believe it'll be a quick fix, so I hope to get it in before Podman 2.0 :)

[vrothberg]

I opened https://github.com/containers/libpod/pull/6615 to fix the issue.

[btannous]

nice! now that was a quick pr. can’t wait to try it out.

[btannous]

now if i can figure out how to use podman in rootless mode with a USB device i’d be all set.

tried a yesterday and failed.
quite a bit of ls -la /dev/bus/usb/001/ inside the container looking like ? ? ? ???

[vrothberg]

@btannous feel free to reach out on IRC, the mailing list or via an issue here on GitHub if you questions or hit issues :)

[btannous]

when i get back into it after work i’ll jump on IRC. :)

[btannous]

Just following up @vrothberg

My device issues using podman rootless were fixed by audit2allow.
grep readsb /var/log/audit/audit.log | audit2allow -a -M mydevicefiltered
then loaded the module like it says and 🥳

[rhatdan]

What were the AVCs you were seeing?

[btannous]

@rhatdan !
This was my first time using audit2allow, but the docs were really helpful so :) there.

[b@adsb adsb]$ cat mydevicefiltered.te #I’d assume this is what you are looking for? 

module mydevicefiltered 1.0;

require {
        type init_t;
        type usb_device_t;
        type admin_home_t;
        type container_t;
        class file { ioctl open read };
        class chr_file { ioctl open read write };
}

#============= container_t ==============
allow container_t usb_device_t:chr_file { ioctl open read write };

#============= init_t ==============
allow init_t admin_home_t:file { ioctl open read };



[b@adsb adsb]$ cat podman.sh # I saved the podman commands I initially ran, these were followed by podman generate systemd....
podman run -d --pull always --label "io.containers.autoupdate=image" --override-arch=arm64 --name adsb-readsb --device /dev/bus/usb/001/004 -p 8080:8080 -p 30005:30005 -e TZ=America/Chicago mikenye/readsb:latest --dcfilter --device-type=rtlsdr --fix --json-location-accuracy=2 --lat=33.12595 --lon=-96.87263 --modeac --ppm=0 --net --stats-every=3600 --quiet --write-json=/run/readsb
podman run -d --pull always --label "io.containers.autoupdate=image" --override-arch=arm64 --name adsb-piaware -e TZ=America/Chicago -e BEASTHOST=192.168.128.13 -e FEEDER_ID=09900bcd-6c92-4d95-ba1a-842956f75288 -e LAT=33.12595 -e LONG=-96.87263  mikenye/piaware:latest 
podman run -d --pull always --label "io.containers.autoupdate=image" --override-arch=arm64 --name adsb-x -e TZ=America/Chicago -e BEASTHOST=192.168.128.13 -e LAT=33.12595 -e LONG=-96.87263 -e ALT=614ft -e SITENAME=quaa00 -e UUID=583c4b5e-af3f-11ea-95b6-da672c6e7d17 mikenye/adsbexchange:latest 

edit: this pi is in my garage hooked up to an RTLSDR (this one https://flightaware.com/adsb/prostick/ ) capturing ADSB and passing it to flightaware and adsbexchange.

edit2: I also had to give me group permissions

[b@adsb ~]$ cat /etc/udev/rules.d/42-usb-permissions.rules 
# /etc/udev/rules.d/42-usb-permissions.rules
SUBSYSTEM!="usb", GOTO="end_skip_usb"
ATTRS{idVendor}=="0bda", ATTRS{idProduct}=="2832", GROUP="b"
LABEL="end_skip_usb"

[btannous]

ha. I feel like I can now properly own this shirt: https://shop.spreadshirt.com/mhayden/setenforce+1-A5d8a6aa81cbf3a25d5e9317b?productType=812&sellable=zrd3xQERz7uLe3JqmeJn-812-7&appearance=2

[vrothberg]

You make Dan very happy with that :)

[rhatdan]

Do you have some kind of terminal that is a usb_device_t?
ls -lZ/dev/ | grep usb_device_t

[btannous]

@rhatdan

[b@adsb ~]$ ls -RlZ /dev/ | grep usb
crw-------. 1 root root    system_u:object_r:usbmon_device_t:s0       238,   0 Apr  1 17:24 usbmon0
crw-------. 1 root root    system_u:object_r:usbmon_device_t:s0       238,   1 Apr  1 17:24 usbmon1
drwxr-xr-x. 3 root root system_u:object_r:device_t:s0 60 Apr  1 17:23 usb
/dev/bus/usb:
/dev/bus/usb/001:
crw-rw-r--. 1 root root system_u:object_r:usb_device_t:s0 189, 0 Apr  1 17:24 001
crw-rw-r--. 1 root root system_u:object_r:usb_device_t:s0 189, 1 Apr  1 17:24 002
crw-rw-r--. 1 root root system_u:object_r:usb_device_t:s0 189, 2 Apr  1 17:24 003
crw-rw-r--. 1 root b    system_u:object_r:usb_device_t:s0 189, 3 Jun 22 17:41 004
crw-rw-r--. 1 root root system_u:object_r:usb_device_t:s0 189, 4 Apr  1 17:24 005
lrwxrwxrwx. 1 root root system_u:object_r:device_t:s0 18 Apr  1 17:24 189:0 -> ../bus/usb/001/001
lrwxrwxrwx. 1 root root system_u:object_r:device_t:s0 18 Apr  1 17:24 189:1 -> ../bus/usb/001/002
lrwxrwxrwx. 1 root root system_u:object_r:device_t:s0 18 Apr  1 17:24 189:2 -> ../bus/usb/001/003
lrwxrwxrwx. 1 root root system_u:object_r:device_t:s0 18 Apr  1 17:24 189:3 -> ../bus/usb/001/004
lrwxrwxrwx. 1 root root system_u:object_r:device_t:s0 18 Apr  1 17:24 189:4 -> ../bus/usb/001/005
lrwxrwxrwx. 1 root root system_u:object_r:device_t:s0 10 Apr  1 17:24 238:0 -> ../usbmon0
lrwxrwxrwx. 1 root root system_u:object_r:device_t:s0 10 Apr  1 17:24 238:1 -> ../usbmon1

[rhatdan]

How did one of those get into the container? Or would you expect the container to access these devices?

[btannous]

I’m passing the device to the container using this:
podman run -d --pull always --label "io.containers.autoupdate=image" --override-arch=arm64 --name adsb-readsb --device /dev/bus/usb/001/004 -p 8080:8080 -p 30005:30005 -e TZ=America/Chicago mikenye/readsb:latest --dcfilter --device-type=rtlsdr --fix --json-location-accuracy=2 --lat=33.12595 --lon=-96.87263 --modeac --ppm=0 --net --stats-every=3600 --quiet --write-json=/run/readsb

tldr; podman .. —device /dev/bus/usb/001/004 ..

And yes, this particular container opens the usb RTLSDR device and listens on the 1090 freq for ADSB decoding using https://github.com/Mictronics/readsb

[rhatdan]

SGTM. nothing I have to change in the defauls. Great work btannous.

In rootfull mode the change policy would not be needed, I believe.

[btannous]

+1 to that, tested and worked in rootfull mode without any changes (ie, without my custom module)

[btannous]

Thanks again for the review!