Runtime: [RFC] Create a candidate list for deprecated feature removal

[mcastelino]

[RFC] Create a candidate list for deprecated feature removal

Today Kata containers has grown to have a lot of features and capabilities. Some of these capabilities are options which a user can choose from. These have evolved over time and newer capabilities have replaced older ones for the same feature. This results in code complexity without additional value. The goal of this issue to identify candidates for removal and leave just the ones that work best for a given hypervisor.

This also increases our validation burden.

This will also allows to default to an implementation for a given hypervisor.

For QEMU/NEMU:

• Networking: Bridge mode, macvtap mode and tcmirror
  - Eliminate Bridge mode as tcmirror replaces it
• Storage: virtio-blk, virtio-scsi and virtio-mmio
  - Eliminate virtio-blk (is there any need for it). It is simpler however.
• VM to Host: vsock, virtio-serial
  - Too early to eliminate virtio-serial but if our user base has newer kernels we can consider eliminating serial

[mcastelino]

/cc @egernst @sameo @sboeuf @gnawux @jodh-intel @amshinde

[mcastelino]

/cc @devimc @jcvenegas

[sboeuf]

:+1: @mcastelino

I agree with the concept of having this list, and also I'm for the removal of bridge and virtio-blk since they're not used and not tested.

I have a different opinion about virtio-serial, since this ensures Kata can be run on older hosts (4.8 IIRC), and I think we still want to be able to claim that no matter the version of your host (to a reasonable point of course), you should be able to run Kata.

[mcastelino]

I have a different opinion about virtio-serial, since this ensures Kata can be run on older hosts (4.8 IIRC), and I think we still want to be able to claim that no matter the version of your host (to a reasonable point of course), you should be able to run Kata.

I agree. The single biggest benefit of only supporting vsock would be the elimination of proxy (which takes out a large component).

If we get feedback from the community if anyone is still using a non vsock enabled host kernel it would be good feature to eliminate.

[devimc]

My two cents:

Agents: hyperstart, kata, noop
Eliminate hyperstart (we haven't supported it since 2.1)

Proxies: cc, kata, noop
Eliminate cc (same as hyperstart)

Shim: cc, kata, noop
Eliminate cc

[caoruidong]

Related https://github.com/kata-containers/runtime/issues/694

[jodh-intel]

Yeah, we have an "embarrassment of riches" ;)

+1 for removing the ol' CC support.

Other considerations:

• Doc updates resulting from this rework:
  
  
  
  • Update @grahamwhaley Big Book of Kata, documenting all features / config options that will be retained after the cull.
  
  
  • https://github.com/clearcontainers/runtime/blob/master/docs/architecture/architecture.md
  
  
• Check if the following also need to be updated:
  
  
  
  • https://github.com/kata-containers/documentation/blob/master/Limitations.md
  
  
  • https://github.com/kata-containers/documentation/blob/master/Developer-Guide.md
  
  

How about creating a ticklist (like @alicefr's awesome https://github.com/kata-containers/kata-containers/issues/33) or a github project so we can track the individual work items maybe?

/cc @egernst.

[grahamwhaley]

@mcastelino @sboeuf wrt vsock and kernel versions, I have a feeling there is at least one major cloud customer who has an old kernel, and are v.unlikely to (be able to) upgrade. IIRC, it was a 3.x era kernel. I'll see if I can find any sort of ref.
I also dug up an old item/comment that notes some older kernel versions for some LTS distro releases - https://github.com/kata-containers/packaging/issues/55#issuecomment-418373855
My gut feeling is we may not be able to drop virtio-serial for some time.

[egernst]

re: vsock and <4.8 support: my gut reaction is that if they really care about security (and are looking at Kata), they probably shouldn't use such an old kernel.

Having said that... if their host is that old - -maybe a VM isolated container is a good idea.

I'd like to get more feedback on this. @gnawux @WeiZhang555 @jon , what are your thoughts here?

[bergwolf]

re: vsock and <4.8 support: my gut reaction is that if they really care about security (and are looking at Kata), they probably shouldn't use such an old kernel.

Many distribution users are still using old kernels and even for those who stick with upstream stable releases, 4.4.169 is still maintained and I'd expect there are many users of it. So I'd suggest that we keep virtio-serial while using vsock whenever it is available. And w.r.t. removing proxy, shimv2 also removes it even when being used with virtio-serial. So it is not a must to drop virtio-serial in order to get rid of kata proxy.

[mcastelino]

And w.r.t. removing proxy, shimv2 also removes it even when being used with virtio-serial. So it is not a must to drop virtio-serial in order to get rid of kata proxy.

@bergwolf yes our goal is proxy elimination and not really serial elimination. So yes, given that we will move to shimv2 we should then only support serial for the shimv2 use case. For v1 people can use vsock if they need to. So the people on older kernels should use newer kata when available, which drops proxy support.

[sameo]

My two cents:

Agents: hyperstart, kata, noop
Eliminate hyperstart (we haven't supported it since 2.1)

Proxies: cc, kata, noop
Eliminate cc (same as hyperstart)

Shim: cc, kata, noop
Eliminate cc

I'm in favour of removing all those. Now if we do remove them, we're left with a kata and a noop implementation for all those. I can bet that the noop only reason to exist is for unit testing. In those cases we should consider entirely removing the noop implementation and using the kata one for unit testing. That allows for:

• Simplification: We can get rid of one abstraction layer, the interface itself.
• Making our unit testing more relevant by using an actual implementation rather than the noop one.

[WeiZhang555]

I'm OK with removing most components above, except virtio-serial, @sameo and @bergwolf have done good summary.

When we're trying to remove these features, one more important question is:

Is there anyone using it? When we remove the feature, will it make someone angry?

Removing codes from current base can always bring risk of breaking something, I think a safe way to do this is, we mark the feature "deprecated" but still keep it in current code for two (or more) releases, print warnings when users are using it, then finally do real deletion when there's nobody jumps up to oppose the deletion.

So far I don't think we have many users, so if you want to delete some codes, do it ASAP before someone put it in real production :smile:

[gnawux]

do it ASAP before someone put it in real production

@WeiZhang555 Agree, deprecate out-of-date features as soon as possible to prevent from confusing newcomers.

[jodh-intel]

I tend to agree. The problem is we don't know if users are already relying on particular features in a production scenario.

"At some point", we're going to need to handle removing deprecated features more carefully: announce that "feature X"...

• Will become deprecated in Kata version A.B.C.
• Will be removed in Kata version A.${foo}.C.

We use semver across all Kata developed components. Conveniently the semver specification explains what we need to do to deprecate features:

• https://semver.org/#how-should-i-handle-deprecating-functionality

---

This got me thinking about @WeiZhang555's https://github.com/kata-containers/runtime/pull/1227's "experimental feature" PR...

| Feature type | Stable | When will it be removed from Kata? |
|-|-|-|
| experimental | no | could be removed at any time (or may become a standard feature) |
| deprecated | yes | at a definite time |

Clearly, there is similarity there. So, we could potentially refactor the experimental feature once it lands to allow users to query which features are going to be removed at some future date:

$ kata-runtime kata-env | grep -i deprecated

Ideally, that output would show the following for deprecated features:

• Feature name.
• Feature description.
• The semver (future) release version when the feature will be removed.
• Maybe even the semver version when the feature was originally introduced?

Since the output of kata-env is machine readable, that could be quite useful for admins and the handling of deprecated features becomes self-documenting. We should still highlight features that will be deprecated at a future date when we make a release though:

$ kata-runtime kata-env | grep -i deprecated >> ./release-notes.txt

[WeiZhang555]

@jodh-intel Nice summary :handshake:

[jiangliu]

We still have dependency on virtio-serial, thanks!

[jcvenegas]

@devimc @mcastelino @sboeuf @jodh-intel just have systemd or agent as init will simply our testing.

[jodh-intel]

Related to systemd as init or not: https://github.com/kata-containers/documentation/wiki/Kata-images

[gabibeyer]

I've noticed virtc doesn't seem to be working at the moment, and uses the hyperstart and cc implementations rather heavily. After talking with @sboeuf it may not be the best ROI to fix it. How would you all feel about including it as a candidate for removal?