Trident: Trident failed to update to 21.01.0

[uberspot]

Describe the bug
I applied the new bundle.yaml and crd_post1.14 to upgrade to 21.01.0 and got an error in the operator:

time="2021-01-31T10:19:42Z" level=error msg="error syncing 'trident/trident-csi': reconcile failed; error re-installing Trident 'trident' ; err: reconcile failed; unable to create RBAC objects while verifying Trident version; err: could not create the Trident cluster role; could not patch Trident Cluster role;  \"\" is invalid: patch: 

Invalid value: \"{\\\"apiVersion\\\":\\\"authorization.openshift.io/v1\\\",\\\"kind\\\":\\\"ClusterRole\\\",\\\"metadata\\\":{\\\"creationTimestamp\\\":\\\"2021-01-31T10:17:12Z\\\",\\\"labels\\\":{\\\"app\\\":\\\"controller.csi.trident.netapp.io\\\",\\\"k8s_version\\\":\\\"v1.19.0\\\"},\\\"managedFields\\\":[{\\\"apiVersion\\\":\\\"rbac.authorization.k8s.io/v1\\\",\\\"fieldsType\\\":\\\"FieldsV1\\\",\\\"fieldsV1\\\":{\\\"f:metadata\\\":{\\\"f:labels\\\":{\\\".\\\":{},\\\"f:app\\\":{},\\\"f:k8s_version\\\":{}},\\\"f:ownerReferences\\\":{\\\".\\\":{},\\\"k:{\\\\\\\"uid\\\\\\\":\\\\\\\"abf1fe5d-372a-4e84-804a-41bbf94cfba6\\\\\\\"}\\\":{\\\".\\\":{},\\\"f:apiVersion\\\":{},\\\"f:controller\\\":{},\\\"f:kind\\\":{},\\\"f:name\\\":{},\\\"f:uid\\\":{}}}},\\\"f:rules\\\":{}},\\\"manager\\\":\\\"openshift-apiserver\\\",\\\"operation\\\":\\\"Update\\\",\\\"time\\\":\\\"2021-01-31T10:17:12Z\\\"}],\\\"name\\\":\\\"trident-csi\\\",\\\"ownerReferences\\\":[{\\\"apiVersion\\\":\\\"trident.netapp.io/v1\\\",\\\"controller\\\":true,\\\"kind\\\":\\\"TridentOrchestrator\\\",\\\"name\\\":\\\"trident\\\",\\\"uid\\\":\\\"abf1fe5d-372a-4e84-804a-41bbf94cfba6\\\"}],\\\"resourceVersion\\\":\\\"340885946\\\",\\\"selfLink\\\":\\\"/apis/rbac.authorization.k8s.io/v1/clusterroles/trident-csi\\\",\\\"uid\\\":\\\"9f336f34-3551-4aac-9662-c083cc0f915a\\\"},\\\"rules\\\":[{\\\"apiGroups\\\":[\\\"\\\"],\\\"resources\\\":[\\\"namespaces\\\"],\\\"verbs\\\":[\\\"get\\\",\\\"list\\\"]},{\\\"apiGroups\\\":[\\\"\\\"],\\\"resources\\\":[\\\"persistentvolumes\\\",\\\"persistentvolumeclaims\\\"],\\\"verbs\\\":[\\\"get\\\",\\\"list\\\",\\\"watch\\\",\\\"create\\\",\\\"delete\\\",\\\"update\\\",\\\"patch\\\"]},{\\\"apiGroups\\\":[\\\"\\\"],\\\"resources\\\":[\\\"persistentvolumeclaims/status\\\"],\\\"verbs\\\":[\\\"update\\\",\\\"patch\\\"]},{\\\"apiGroups\\\":[\\\"storage.k8s.io\\\"],\\\"resources\\\":[\\\"storageclasses\\\"],\\\"verbs\\\":[\\\"get\\\",\\\"list\\\",\\\"watch\\\",\\\"create\\\",\\\"delete\\\",\\\"update\\\",\\\"patch\\\"]},{\\\"apiGroups\\\":[\\\"\\\"],\\\"resources\\\":[\\\"events\\\"],\\\"verbs\\\":[\\\"get\\\",\\\"list\\\",\\\"watch\\\",\\\"create\\\",\\\"update\\\",\\\"patch\\\"]},{\\\"apiGroups\\\":[\\\"\\\"],\\\"resources\\\":[\\\"secrets\\\"],\\\"verbs\\\":[\\\"get\\\",\\\"list\\\",\\\"watch\\\",\\\"create\\\",\\\"delete\\\",\\\"update\\\",\\\"patch\\\"]},{\\\"apiGroups\\\":[\\\"\\\"],\\\"resources\\\":[\\\"pods\\\"],\\\"verbs\\\":[\\\"get\\\",\\\"list\\\",\\\"watch\\\",\\\"create\\\",\\\"delete\\\",\\\"update\\\",\\\"patch\\\"]},{\\\"apiGroups\\\":[\\\"\\\"],\\\"resources\\\":[\\\"pods/log\\\"],\\\"verbs\\\":[\\\"get\\\",\\\"list\\\",\\\"watch\\\"]},{\\\"apiGroups\\\":[\\\"\\\"],\\\"resources\\\":[\\\"nodes\\\"],\\\"verbs\\\":[\\\"get\\\",\\\"list\\\",\\\"watch\\\",\\\"update\\\"]},{\\\"apiGroups\\\":[\\\"storage.k8s.io\\\"],\\\"resources\\\":[\\\"volumeattachments\\\"],\\\"verbs\\\":[\\\"get\\\",\\\"list\\\",\\\"watch\\\",\\\"update\\\",\\\"patch\\\"]},{\\\"apiGroups\\\":[\\\"storage.k8s.io\\\"],\\\"resources\\\":[\\\"volumeattachments/status\\\"],\\\"verbs\\\":[\\\"update\\\",\\\"patch\\\"]},{\\\"apiGroups\\\":[\\\"snapshot.storage.k8s.io\\\"],\\\"resources\\\":[\\\"volumesnapshots\\\",\\\"volumesnapshotclasses\\\"],\\\"verbs\\\":[\\\"get\\\",\\\"list\\\",\\\"watch\\\",\\\"update\\\",\\\"patch\\\"]},{\\\"apiGroups\\\":[\\\"snapshot.storage.k8s.io\\\"],\\\"resources\\\":[\\\"volumesnapshots/status\\\",\\\"volumesnapshotcontents/status\\\"],\\\"verbs\\\":[\\\"update\\\",\\\"patch\\\"]},{\\\"apiGroups\\\":[\\\"snapshot.storage.k8s.io\\\"],\\\"resources\\\":[\\\"volumesnapshotcontents\\\"],\\\"verbs\\\":[\\\"get\\\",\\\"list\\\",\\\"watch\\\",\\\"create\\\",\\\"delete\\\",\\\"update\\\",\\\"patch\\\"]},{\\\"apiGroups\\\":[\\\"csi.storage.k8s.io\\\"],\\\"resources\\\":[\\\"csidrivers\\\",\\\"csinodeinfos\\\"],\\\"verbs\\\":[\\\"get\\\",\\\"list\\\",\\\"watch\\\",\\\"create\\\",\\\"delete\\\",\\\"update\\\",\\\"patch\\\"]},{\\\"apiGroups\\\":[\\\"storage.k8s.io\\\"],\\\"resources\\\":[\\\"csidrivers\\\",\\\"csinodes\\\"],\\\"verbs\\\":[\\\"get\\\",\\\"list\\\",\\\"watch\\\",\\\"create\\\",\\\"delete\\\",\\\"update\\\",\\\"patch\\\"]},{\\\"apiGroups\\\":[\\\"apiextensions.k8s.io\\\"],\\\"resources\\\":[\\\"customresourcedefinitions\\\"],\\\"verbs\\\":[\\\"get\\\",\\\"list\\\",\\\"watch\\\",\\\"create\\\",\\\"delete\\\",\\\"update\\\",\\\"patch\\\"]},{\\\"apiGroups\\\":[\\\"trident.netapp.io\\\"],\\\"resources\\\":[\\\"tridentversions\\\",\\\"tridentbackends\\\",\\\"tridentstorageclasses\\\",\\\"tridentvolumes\\\",\\\"tridentnodes\\\",\\\"tridenttransactions\\\",\\\"tridentsnapshots\\\"],\\\"verbs\\\":[\\\"get\\\",\\\"list\\\",\\\"watch\\\",\\\"create\\\",\\\"delete\\\",\\\"update\\\",\\\"patch\\\"]},{\\\"apiGroups\\\":[\\\"policy\\\"],\\\"resourceNames\\\":[\\\"tridentpods\\\"],\\\"resources\\\":[\\\"podsecuritypolicies\\\"],\\\"verbs\\\":[\\\"use\\\"]}]}\": 

no kind \"ClusterRole\" is registered for version \"authorization.openshift.io/v1\" in scheme \"k8s.io/kubernetes/pkg/api/legacyscheme/scheme.go:30\", requeuing"

I believe the rbac suggested usage has changed on 4.6 ? https://docs.openshift.com/container-platform/4.6/authentication/using-rbac.html this should just be a quick yaml fix on the clusterRole.

Environment
Provide accurate information about the environment to help us reproduce the issue.

• Trident version: 20.10.0 -> 21.01.0
• Trident installation flags used: [e.g. -d -n trident --use-custom-yaml]
• Container runtime: [e.g. Docker 19.03.1-CE]
• Kubernetes version: 1.19
• Kubernetes orchestrator: Openshift 4.6

* Suggested fix *

Remove this if case https://github.com/NetApp/trident/blob/b0d57344c5d1e0179308802764270e765427c5c8/cli/k8s_client/yaml_factory.go#L78 ?
Since now it should hopefully be supported on openshift without a custom solution.

[ntap-arorar]

Hello @uberspot

We have investigated this issue and identified that it affects only Red Hat OpenShift Container Platform (OCP) 4.x installations and is due to the deprecated "authorization.openshift.io/v1" API Version. We are actively working on a 21.01.1 hot-fix for this issue.

There is no impact on the Trident installation, it can continue to function normally but it blocks Trident Operator from managing Trident and making modifications to the Trident installation based on the TridentOrchestrator CR changes.

The workaround here is to delete the 'trident-csi' ClusterRole and 'trident-csi' ClusterRoleBinding. This would temporarily unblock the Trident Operator so that it can complete the reconcile, make required changes (based on TridentOrchestrator CR changes) and re-create the ClusterRole and ClusterRoleBinding.

oc delete clusterrole trident-csi
oc delete clusterrolebinding trident-csi

All other installation methods, including tridentctl-based installs and Operator/Helm installs on non-OCP distributions are not impacted. At this time we ask that if you are using OCP and the Trident Operator or Helm that you hold off upgrading until 21.01.1 is released.

[gnarl]

Hi @uberspot,

This issue is now fixed in the Trident v21.01.1 release.

[uberspot]

Rolled it out and confirmed fix is working. :)
Many thanks!