cri-o v1.18.x is not available on Ubuntu?

[brightzheng100]

Description

cri-o v1.18.x is not available on Ubuntu while v1.15, v1.16, v1.17 are.

Steps to reproduce the issue:

Run as root:

. /etc/os-release
sh -c "echo 'deb http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/x${NAME}_${VERSION_ID}/ /' > /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list"
wget -nv https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable/x${NAME}_${VERSION_ID}/Release.key -O- | apt-key add -
apt-get update -qq

apt-cache search cri-o

Describe the results you received:

OUTPUT:

cri-o-1.15 - OCI-based implementation of Kubernetes Container Runtime Interface.
cri-o-1.16 - OCI-based implementation of Kubernetes Container Runtime Interface.
cri-o-1.17 - OCI-based implementation of Kubernetes Container Runtime Interface.
cri-o-runc - Open Container Project - runtime

Describe the results you expected:

The doc has indicated that all desired versions should be available to install.
I'm expecting v1.18 at this moment.

Additional environment details (AWS, VirtualBox, physical, etc.):

$ cat /etc/os-release
NAME="Ubuntu"
VERSION="18.04.4 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.4 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

[haircommander]

AFAIU it's because OBS on ubuntu hasn't gotten proper support for the required go version. Is that correct @dougsland

[dougsland]

AFAIU it's because OBS on ubuntu hasn't gotten proper support for the required go version. Is
that correct @dougsland

CRI-O 1.18 requires go 1.13 to compile. Thus, we are unable to support building on operating systems that do not have go 1.13. Users can always build golang/crio manually.

[mrunalp]

Ca we setup a ppa for 1.18 meanwhile?

[afbjorklund]

Ubuntu 20.04 has go 1.13 support

[haircommander]

Ubuntu 20.04 has go 1.13 support

and we currently support it with cri-o 1.18 :)

[afbjorklund]

For some reason it is not showing up in the package list for Ubuntu 20.04

E: Unable to locate package cri-o-1.18

Home page still suggests to use previous, but I guess that is for 16.04/18.04

CRIO_VERSION=1.17

[afbjorklund]

Something is definitely weird:

$ apt search cri-o
Sorting... Done
Full Text Search... Done
cri-o-1.15/unknown 1.15.4-1~dev~ubuntu19.04~ppa4 s390x
  OCI-based implementation of Kubernetes Container Runtime Interface.

cri-o-1.16/unknown 1.16.6~1 s390x
  OCI-based implementation of Kubernetes Container Runtime Interface.

cri-o-1.17/unknown 1.17.4~1 s390x
  OCI-based implementation of Kubernetes Container Runtime Interface.

cri-o-runc/unknown 1.0.0-3~dev2 s390x
  Open Container Project - runtime

Supposed to be amd64...

$ cat /etc/apt/sources.list.d/devel\:kubic\:libcontainers\:stable.list 
deb http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/xUbuntu_20.04/ /

Can't see the deb file for 1.18 anywhere on the site, either ?

http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/xUbuntu_20.04/

So I'm not sure about "supported", did it test OK somewhere ?

Only in: https://build.opensuse.org/project/show/devel:kubic:libcontainers:stable:cri-o:1.18

The cri-o-1.17 package did install OK.

$ apt list cri-o-1.17
Listing... Done
cri-o-1.17/unknown,now 1.17.4~1 amd64 [installed]
cri-o-1.17/unknown 1.17.4~1 arm64
cri-o-1.17/unknown 1.17.4~1 armhf
cri-o-1.17/unknown 1.17.4~1 s390x
$ crio --version
crio version 1.17.4
$ grep PRETTY /etc/os-release 
PRETTY_NAME="Ubuntu 20.04 LTS"

[afbjorklund]

CRI-O 1.18 requires go 1.13 to compile. Thus, we are unable to support building on operating systems that do not have go 1.13. Users can always build golang/crio manually.

"Users can always CRI-O 1.17", is more likely to happen. When you bump the go requirements ahead of the system, I mean... Kubernetes is using go1.13.8, but the OS are older

[afbjorklund]

@haircommander : it would be great if this could be released next week, then it could go into minikube 1.12 (scheduled for end of June). We have upgraded ubuntu from 19.10 to 20.04, but still cri-o-1.17

apt-get install -y --no-install-recommends cri-o-1.17

See https://github.com/kubernetes/minikube/pull/8251

Also had to pin podman due to another issue (with build), but that should be fixed now so I think we are ready to go to 1.9.3~1 - we use it for loading images from cache files since crictl doesn't support that

apt-get install -y --no-install-recommends podman=1.8.2~2

See https://github.com/kubernetes/minikube/issues/8063

[afbjorklund]

Here is my rebuild of the package:
https://bintray.com/afbjorklund/cri-o/

For some reason it is called cri-o rather than cri-o-1.18, and I only needed it for focal (20.04)
I was rebuilding the sources from the "1.18" branch, in order to upgrade from 1.17.4 to 1.18.1.

https://build.opensuse.org/package/show/devel:kubic:libcontainers:stable:cri-o:1.18/cri-o

[Exagone313]

Building and running are two different things, unless there is a hard requirement on libc? Package manager could also provide static builds built using Nix.

CRI-O builds well on Ubuntu 20, but it is not straight forward to create a package on this distribution, and not having a package for it prevents installing Kubernetes from their official repository if you don't have a recognized container runtime package. (Official Kubernetes prebuilts are working too, but it's a manual installation.)

@afbjorklund Are you able to install Kubernetes from their repository with your package installed? Can you provide build instructions? Thanks!

[afbjorklund]

Mixing and matching APT and Nix seems even worse than mixing system packages (ubuntu) and vendor packages (opensuse)

I _think_ there are static binaries available that you could use, but putting those into deb packages is only making this worse IMHO

https://github.com/cri-o/cri-o/releases/tag/v1.18.1

Download the static release bundle via our Google Cloud Bucket:
crio-v1.18.1.tar.gz

---

Here were my build instructions:

sudo pbuilder create --basetgz /var/cache/pbuilder/focal.tgz --distribution focal
sudo pbuilder build --basetgz /var/cache/pbuilder/focal.tgz --distribution focal cri-o_1.18.1~1.dsc

It _seemed_ to install OK (as cri-o)

https://bintray.com/afbjorklund/cri-o/cri-o/1.18.1

But I would prefer if there was one APT source, and don't _really_ want to maintain podman cri-o packages downstream...

Maybe should give the crio-static a chance, that is the approach that KIND is using when it comes to e.g. containerd.

[afbjorklund]

https://bintray.com/afbjorklund/cri-o/cri-o/1.18.2 (upgrade from 1.17.4)

https://bintray.com/afbjorklund/podman/podman/1.9.3 (downgrade from 2.0.1)

[haircommander]

1.18.3 is now being built successfully in stable:
https://build.opensuse.org/project/show/devel:kubic:libcontainers:stable:cri-o:1.18:1.18.3

[inductor]

I still cannot find the packages... :(

[haircommander]

turns out, only debian has succeeded so far, and not the older ubuntu versions... sorry folks, we're still working on it!

[haircommander]

ok we've been having more success so far. I believe 1.18 and 1.18:1.18.3 should be ready soon. Thanks for the patience everyone!

[afbjorklund]

Updating CRI-O in minikube was pushed to september (1.14), so most likely next release will have k8s 1.19 with crio 1.17

See https://github.com/kubernetes/minikube/issues/8388 (had to be reverted) which is blocked by https://github.com/kubernetes/minikube/issues/7904 (for go 1.13 support)

[haircommander]

CRI-O 1.18.x is now available on ubuntu < 20.04
https://build.opensuse.org/package/show/devel:kubic:libcontainers:stable:cri-o:1.18/cri-o
as such, I am closing this. if there's an issue with the builds, please let me know :smile:

[afbjorklund]

As noted above, you need to make it available as “crio-1.18” in the main repo or it won’t be visible to users

http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/xUbuntu_20.04/

[haircommander]

I disagree. If we correctly document the location (https://github.com/cri-o/cri-o/pull/3901), then it shouldn't matter where we put them. the top level crio-1.x are considered legacy now, as we're using subprojects now instead. Once each of those versions are EOL, they will be cleaned up as well.

[inductor]

The main reason I reached at this issue is https://kubernetes.io/docs/setup/production-environment/container-runtimes/#cri-o

I think the CRI-O community should fix this installation

[afbjorklund]

I disagree. If we correctly document the location (#3901), then it shouldn't matter where we put them

Sure, as long as there is a reasonable documentation for all the "xubuntu" and other quirks it is fine.

Right now we have https://cri-o.io/

 CRIO_VERSION=1.17 
  . /etc/os-release
  sudo sh -c "echo 'deb http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/x${NAME}_${VERSION_ID}/ /' >/etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list"
  wget -nv https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable/x${NAME}_${VERSION_ID}/Release.key -O- | sudo apt-key add -
  sudo apt-get update -qq
  sudo apt-get install cri-o-${CRIO_VERSION}

and https://github.com/cri-o/cri-o

# Ubuntu (18.04, 19.04, 19.10, and 20.04)
. /etc/os-release
sudo sh -c "echo 'deb http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/x${NAME}_${VERSION_ID}/ /' > /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list"
wget -nv https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable/x${NAME}_${VERSION_ID}/Release.key -O- | sudo apt-key add -

sudo apt-get update -qq
sudo apt-get install cri-o-[REQUIRED VERSION]

And the official https://kubernetes.io/docs/setup/production-environment/container-runtimes/#cri-o

# Configure package repository
. /etc/os-release
sudo sh -c "echo 'deb http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/x${NAME}_${VERSION_ID}/ /' > /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list"
wget -nv https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable/x${NAME}_${VERSION_ID}/Release.key -O- | sudo apt-key add -
sudo apt-get update

# Install CRI-O
sudo apt-get install cri-o-1.17

So it seems to be rather aligned. Maybe needs the usual blurb on how to enable https repositories.

[haircommander]

yes I agree @afbjorklund @inductor, and thanks for raising awareness of these locations. Once https://github.com/cri-o/cri-o/pull/3901 merges, I'll mirror it to those locations. in the mean time, I'd love reviews on it :smile:

[afbjorklund]

Will prepare another upgrade, but for now all upgrades to the OS were postponed until next release.

Troubleshooting some other crio issues meanwhile, which all seem to come from "not being Docker"

https://github.com/kubernetes/minikube/issues/8919

https://github.com/kubernetes/minikube/labels/co%2Fruntime%2Fcrio

[haircommander]

k8s doc update: https://github.com/kubernetes/website/pull/23000
cri-o.io docs update: https://github.com/cri-o/cri-o.io/pull/45