Linkerd2: Add support for k8s 1.19

[alpeb]

It appears that with the latest edge everything works fine with k8s 1.19 except for the warnings about the resource versions deprecations, e.g. in linkerd check we see things like this in stderr:

W0106 11:09:39.204081  292603 warnings.go:67] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition

This should be just a matter of updating the versions in the template files, along with the tests golden files.

Note that as of #5483 bin/k3d already installs k8s 1.19 by default.

[mateiidavid]

👋 I'd like to take this on if that's ok!

[kleimkuhler]

That'd be great @mateiidavid!

[cpretzer]

Thanks @mateiidavid !

[mateiidavid]

Hi, I started working on this yesterday. I think there may be a bit more work than initially thought.

What I have done at first was to change all .golden files and all templates to use v1 instead of v1beta1 for apiextensions.k8s.io and admissionregistration.k8s.io resources.

To test, I tried a fresh install on my k3s cluster. Unfortunately, warnings were replaced with errors 😢:

error validating "STDIN": error validating data: [ValidationError(CustomResourceDefinition.spec): unknown field "additionalPrinterColumns" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec, ValidationError(CustomResourceDefinition.spec): unknown field "version" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec, ValidationError(CustomResourceDefinition.spec): missing required field "versions" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec]; if you choose to ignore these errors, turn validation off with --validate=false
Error from server (Invalid): error when creating "STDIN": CustomResourceDefinition.apiextensions.k8s.io "serviceprofiles.linkerd.io" is invalid: [spec.versions[0].schema.openAPIV3Schema: Required value: schemas are required, spec.versions[1].schema.openAPIV3Schema: Required value: schemas are required]

I think just changing the apiVersion on the resources might not be enough.

• For CRDs, the official docs state that _"With apiextensions.k8s.io/v1 the definition of a structural schema is mandatory for CustomResourceDefinitions. In the beta version of CustomResourceDefinition, the structural schema was optional."_. As far as I can tell we'd have to explicitly define an openAPIv3Schema field for all CRDs (McLink, TrafficSplit & SP).
• For admission webhooks, as of v1, admissionReviewVersions is a required field. Again this means changing all admission webhooks to include this field (which I've already done; it will accept [v1, v1beta1]).
• When we fetch resources in uninstall.go we fetch them using client-go. The issue here is that we use versioned clientsets (v1beta1 in both cases). If I understand this pragraph correctly we get warnings here because we consume the wrong version of the API (?). I changed the clientset/library version to v1 in both cases and it fixed the warnings.

# uninstall warnings
 bin/go-run cli uninstall | k delete -f -
W0113 12:38:17.323234   94040 warnings.go:67] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
W0113 12:38:17.353919   94040 warnings.go:67] admissionregistration.k8s.io/v1beta1 MutatingWebhookConfiguration is deprecated in v1.16+, unavailable in v1.22+; use admissionregistration.k8s.io/v1 MutatingWebhookConfiguration
W0113 12:38:17.362157   94040 warnings.go:67] admissionregistration.k8s.io/v1beta1 ValidatingWebhookConfiguration is deprecated in v1.16+, unavailable in v1.22+; use admissionregistration.k8s.io/v1 ValidatingWebhookConfiguration
clusterrole.rbac.authorization.k8s.io "linkerd-linkerd-identity" deleted
clusterrole.rbac.authorization.k8s.io "linkerd-linkerd-controller" deleted
clusterrole.rbac.authorization.k8s.io "linkerd-linkerd-destination" deleted
clusterrolebinding.rbac.authorization.k8s.io "linkerd-linkerd-identity" deleted
clusterrolebinding.rbac.authorization.k8s.io "linkerd-linkerd-controller" deleted
clusterrolebinding.rbac.authorization.k8s.io "linkerd-linkerd-destination" deleted

I think this also applies to healthcheck.go (?), I took a quick look and we're using v1beta1 when working with these resources (e.g in fetchSpValidatorCaBundle). I have changed both healthcheck.go and uninstall.go but I am unsure whether this is required or not -- I'd rather not break more code than I fix 😅 .

Anyway, I thought before making any further changes to quickly get some input and see if I'm on the right track. Are there any particular opinions here? I'd like to know if first of all _I'm wrong with my assumptions_. Next, I'd like to see if you are okay with me making such a big change, perhaps the go files can be changed in a subsequent PR, or maybe we can leave the CRDs for a separate PR seeing as they require the explicit schema. In short just want to know wdyt @alpeb @kleimkuhler @cpretzer

[alpeb]

Hi @mateiidavid :wave:

I have no detailed knowledge of how versioning here works, but from what I can tell, moving to v1, including the client-go methods, it seems to be fine. IMO verifying the CLI remains sane when hitting say a linkerd stable-2.9.1 installation should be enough, along with making sure things continue to work with k8s 1.15.
As for CRD schema, yeah, it might be better to change that first in a separate PR.