Okd: KubeVirt support

[jdhirst]

Looking to add semanage binary to fcos so as to allow Kubevirt and other similar solutions to manage selinux labels.

Currently kubevirt has this PR: https://github.com/kubevirt/kubevirt/pull/2694

However, it would be useful to have semanage in general.

[vrutkovs]

semanage requires python dependencies, which means we'd have to pull in a lot of RPMs one by one and carefully track dependencies. I'm afraid that would be too complicated to manage

[jdhirst]

This problem must have been solved in RHCOS in order to enable openshift-cnv, as it has the same problem. (tested openshift-cnv 2.2 on okd4.4/fcos)

{"component":"virt-handler","hostname":"master-2.okd.example.net","level":"info","pos":"virt-handler.go:187","timestamp":"2020-02-10T16:56:45.794105Z"}
panic: failed to list labels: out: "", error: could not find binary semanage

goroutine 1 [running]:
main.(*virtHandlerApp).Run(0xc000246000)
    cmd/virt-handler/virt-handler.go:205 +0x1ea7
main.main()
    cmd/virt-handler/virt-handler.go:468 +0x6e

[vrutkovs]

Right, RHCOS does include Python and semanage RPMs. However in FCOS-on-OKD4 we'd like to avoid that (services should not rely on specifics of host utils - and we mix in RPMs one by one at the moment, so tracking dependencies would be complex).

rpm-ostree install <rpm name> however would work, so that's probably the right way to go about this

[LorbusChris]

I believe something like https://github.com/kubevirt/kubevirt/pull/2694 would be a proper solution for this

[LorbusChris]

I've renamed this issue - we're not going to include semanage in Fedora or OKD's machine-os-content by default.

[rmohr]

Note that apart from relabeling we also need to install our own policy, which obviously also does not work in rhcos and fcos. I will redbase https://github.com/kubevirt/kubevirt/pull/2694 but users will right now also have to force kubevirt to use spc_t for our workloads (https://github.com/kubevirt/kubevirt/pull/3056). That is of course not an optimal solution, but the best we have right now and you can then at least still leave selinux enabled.

[vladikr]

@rmohr installation of new policy does work on rhcos, as demonstrated in [1] - however,
semodule is required

[1] https://github.com/JAORMX/selinux-operator/blob/master/pkg/controller/configmap/configmap_controller.go#L133

[rmohr]

@rmohr installation of new policy does work on rhcos, as demonstrated in [1] - however,
semodule is required

hm, but we assume semodule and I thought that we fail to install also on rhcos. Would we have to ship it or is it on the node?

[vladikr]

hm, but we assume semodule and I thought that we fail to install also on rhcos. Would we have to ship it or is it on the node?

All I'm saying is that it's possible to install on rhcos. This project ships semodule, hopefully, it will be available in rhcos at some point (so I heard), but until then, if we want to make it work ourselves we will need to ship it...

[vrutkovs]

semodule requires Python, right? We won't ship python on FCOS

[vladikr]

semodule requires Python, right? We won't ship python on FCOS

@vrutkovs semodule is written in C, afaik[1]

[1] https://github.com/SELinuxProject/selinux/tree/master/policycoreutils/semodule

[vrutkovs]

So which RPM should we include in OKD-on-FCOS to make things work?

Note that at this point we have to specify every dependency manually from Fedora 31 repos (list of RPMs already included there) - and we need to add a few more to enable KubeVirt, then it can be done

[LorbusChris]

$ rpm -qf /usr/sbin/semodule
policycoreutils-2.9-5.fc31.x86_64

semodule is included in the policycoreutils rpm, which already ships in FCOS (and I would assume in RHCOS as well)

[LorbusChris]

So depending on semodule would be fine

[jasonbrooks]

I was wondering why kubevirt didn't show up in the OperatorHub in my OKD cluster -- I thought something might be misconfigured, but maybe this is the reason? (may be https://github.com/kubevirt/kubevirt/issues/2932)

[vrutkovs]

I was wondering why kubevirt didn't show up in the OperatorHub in my OKD cluster

Its unlikely to be related - probably it was removed from OperatorHub? In any case operatorhub can't hide operators based on packages installed in the OS

[rajinator]

@jasonbrooks go to Administration->Global Configuration->OperatorHub->Sources and make sure all sources shown in the image are enabled

Edit: When enabled, you'll see kubevirt HCO operator which is named Container Native Virtualization, which usually doesn't show up in OKD as it's not a community operator

[LorbusChris]

@vrutkovs this still needs https://github.com/kubevirt/kubevirt/pull/2694 to merge - it won't work as-is afaict.
/reopen

[vrutkovs]

Moving this out of Beta6 milestone as OKD did its part and we're waiting for updated KubeVirt operator

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[vrutkovs]

/remove-lifecycle stale

[jasonbrooks]

I can confirm that Kubevirt is working for me in OKD now.

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[jdhirst]

Closing as this issue was previously noted as fixed.