Cert-manager: Upgrade from v0.8.1 to v0.9.1 broken

[richstokes]

kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v0.9.1/cert-manager.yaml --validate=false 


customresourcedefinition.apiextensions.k8s.io/certificates.certmanager.k8s.io configured
customresourcedefinition.apiextensions.k8s.io/certificaterequests.certmanager.k8s.io configured
customresourcedefinition.apiextensions.k8s.io/challenges.certmanager.k8s.io configured
customresourcedefinition.apiextensions.k8s.io/clusterissuers.certmanager.k8s.io configured
customresourcedefinition.apiextensions.k8s.io/issuers.certmanager.k8s.io configured
customresourcedefinition.apiextensions.k8s.io/orders.certmanager.k8s.io configured
namespace/cert-manager unchanged
serviceaccount/cert-manager-cainjector unchanged
serviceaccount/cert-manager-webhook unchanged
serviceaccount/cert-manager unchanged
clusterrole.rbac.authorization.k8s.io/cert-manager-cainjector unchanged
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-cainjector unchanged
clusterrole.rbac.authorization.k8s.io/cert-manager-leaderelection unchanged
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-issuers unchanged
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers unchanged
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-certificates unchanged
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-orders unchanged
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-challenges unchanged
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim unchanged
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-leaderelection unchanged
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-issuers unchanged
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers unchanged
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-certificates unchanged
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-orders unchanged
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-challenges unchanged
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim unchanged
clusterrole.rbac.authorization.k8s.io/cert-manager-view unchanged
clusterrole.rbac.authorization.k8s.io/cert-manager-edit unchanged
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-webhook:auth-delegator configured
rolebinding.rbac.authorization.k8s.io/cert-manager-webhook:webhook-authentication-reader configured
clusterrole.rbac.authorization.k8s.io/cert-manager-webhook:webhook-requester unchanged
service/cert-manager-webhook unchanged
apiservice.apiregistration.k8s.io/v1beta1.admission.certmanager.k8s.io unchanged
issuer.certmanager.k8s.io/cert-manager-webhook-selfsign unchanged
certificate.certmanager.k8s.io/cert-manager-webhook-ca unchanged
issuer.certmanager.k8s.io/cert-manager-webhook-ca unchanged
certificate.certmanager.k8s.io/cert-manager-webhook-webhook-tls unchanged
validatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook unchanged
Error from server (Invalid): error when applying patch:
{"metadata":{"annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"apps/v1\",\"kind\":\"Deployment\",\"metadata\":{\"annotations\":{},\"labels\":{\"app\":\"cainjector\",\"app.kubernetes.io/instance\":\"cert-manager\",\"app.kubernetes.io/managed-by\":\"Tiller\",\"app.kubernetes.io/name\":\"cainjector\",\"helm.sh/chart\":\"cainjector-v0.9.1\"},\"name\":\"cert-manager-cainjector\",\"namespace\":\"cert-manager\"},\"spec\":{\"replicas\":1,\"selector\":{\"matchLabels\":{\"app\":\"cainjector\",\"app.kubernetes.io/instance\":\"cert-manager\",\"app.kubernetes.io/managed-by\":\"Tiller\",\"app.kubernetes.io/name\":\"cainjector\"}},\"template\":{\"metadata\":{\"annotations\":null,\"labels\":{\"app\":\"cainjector\",\"app.kubernetes.io/instance\":\"cert-manager\",\"app.kubernetes.io/managed-by\":\"Tiller\",\"app.kubernetes.io/name\":\"cainjector\",\"helm.sh/chart\":\"cainjector-v0.9.1\"}},\"spec\":{\"containers\":[{\"args\":[\"--v=2\",\"--leader-election-namespace=$(POD_NAMESPACE)\"],\"env\":[{\"name\":\"POD_NAMESPACE\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"metadata.namespace\"}}}],\"image\":\"quay.io/jetstack/cert-manager-cainjector:v0.9.1\",\"imagePullPolicy\":\"IfNotPresent\",\"name\":\"cainjector\",\"resources\":{}}],\"serviceAccountName\":\"cert-manager-cainjector\"}}}}\n"},"labels":{"app.kubernetes.io/instance":"cert-manager","app.kubernetes.io/managed-by":"Tiller","app.kubernetes.io/name":"cainjector","chart":null,"helm.sh/chart":"cainjector-v0.9.1","heritage":null,"release":null}},"spec":{"selector":{"matchLabels":{"app.kubernetes.io/instance":"cert-manager","app.kubernetes.io/managed-by":"Tiller","app.kubernetes.io/name":"cainjector","release":null}},"template":{"metadata":{"annotations":null,"labels":{"app.kubernetes.io/instance":"cert-manager","app.kubernetes.io/managed-by":"Tiller","app.kubernetes.io/name":"cainjector","helm.sh/chart":"cainjector-v0.9.1","release":null}},"spec":{"$setElementOrder/containers":[{"name":"cainjector"}],"containers":[{"image":"quay.io/jetstack/cert-manager-cainjector:v0.9.1","name":"cainjector"}]}}}}
to:
Resource: "apps/v1, Resource=deployments", GroupVersionKind: "apps/v1, Kind=Deployment"
Name: "cert-manager-cainjector", Namespace: "cert-manager"
Object: &{map["apiVersion":"apps/v1" "kind":"Deployment" "metadata":map["annotations":map["deployment.kubernetes.io/revision":"1" "kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"apps/v1beta1\",\"kind\":\"Deployment\",\"metadata\":{\"annotations\":{},\"labels\":{\"app\":\"cainjector\",\"chart\":\"cainjector-v0.8.1\",\"heritage\":\"Tiller\",\"release\":\"cert-manager\"},\"name\":\"cert-manager-cainjector\",\"namespace\":\"cert-manager\"},\"spec\":{\"replicas\":1,\"selector\":{\"matchLabels\":{\"app\":\"cainjector\",\"release\":\"cert-manager\"}},\"template\":{\"metadata\":{\"annotations\":null,\"labels\":{\"app\":\"cainjector\",\"release\":\"cert-manager\"}},\"spec\":{\"containers\":[{\"args\":[\"--v=2\",\"--leader-election-namespace=$(POD_NAMESPACE)\"],\"env\":[{\"name\":\"POD_NAMESPACE\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"metadata.namespace\"}}}],\"image\":\"quay.io/jetstack/cert-manager-cainjector:v0.8.1\",\"imagePullPolicy\":\"IfNotPresent\",\"name\":\"cainjector\",\"resources\":{}}],\"serviceAccountName\":\"cert-manager-cainjector\"}}}}\n"] "creationTimestamp":"2019-07-16T17:56:59Z" "generation":'\x01' "labels":map["app":"cainjector" "chart":"cainjector-v0.8.1" "heritage":"Tiller" "release":"cert-manager"] "name":"cert-manager-cainjector" "namespace":"cert-manager" "resourceVersion":"9217853" "selfLink":"/apis/apps/v1/namespaces/cert-manager/deployments/cert-manager-cainjector" "uid":"1bc2acfe-a7f3-11e9-bc98-0629cdf86a62"] "spec":map["progressDeadlineSeconds":'\u0258' "replicas":'\x01' "revisionHistoryLimit":'\x02' "selector":map["matchLabels":map["app":"cainjector" "release":"cert-manager"]] "strategy":map["rollingUpdate":map["maxSurge":"25%" "maxUnavailable":"25%"] "type":"RollingUpdate"] "template":map["metadata":map["creationTimestamp":<nil> "labels":map["app":"cainjector" "release":"cert-manager"]] "spec":map["containers":[map["args":["--v=2" "--leader-election-namespace=$(POD_NAMESPACE)"] "env":[map["name":"POD_NAMESPACE" "valueFrom":map["fieldRef":map["apiVersion":"v1" "fieldPath":"metadata.namespace"]]]] "image":"quay.io/jetstack/cert-manager-cainjector:v0.8.1" "imagePullPolicy":"IfNotPresent" "name":"cainjector" "resources":map[] "terminationMessagePath":"/dev/termination-log" "terminationMessagePolicy":"File"]] "dnsPolicy":"ClusterFirst" "restartPolicy":"Always" "schedulerName":"default-scheduler" "securityContext":map[] "serviceAccount":"cert-manager-cainjector" "serviceAccountName":"cert-manager-cainjector" "terminationGracePeriodSeconds":'\x1e']]] "status":map["availableReplicas":'\x01' "conditions":[map["lastTransitionTime":"2019-07-16T17:56:59Z" "lastUpdateTime":"2019-07-16T17:57:01Z" "message":"ReplicaSet \"cert-manager-cainjector-744b987848\" has successfully progressed." "reason":"NewReplicaSetAvailable" "status":"True" "type":"Progressing"] map["lastTransitionTime":"2019-08-06T08:02:43Z" "lastUpdateTime":"2019-08-06T08:02:43Z" "message":"Deployment has minimum availability." "reason":"MinimumReplicasAvailable" "status":"True" "type":"Available"]] "observedGeneration":'\x01' "readyReplicas":'\x01' "replicas":'\x01' "updatedReplicas":'\x01']]}
for: "https://github.com/jetstack/cert-manager/releases/download/v0.9.1/cert-manager.yaml": Deployment.apps "cert-manager-cainjector" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app.kubernetes.io/name":"cainjector", "app.kubernetes.io/instance":"cert-manager", "app":"cainjector", "app.kubernetes.io/managed-by":"Tiller"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable
Error from server (Invalid): error when applying patch:
{"metadata":{"annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"apps/v1\",\"kind\":\"Deployment\",\"metadata\":{\"annotations\":{},\"labels\":{\"app\":\"webhook\",\"app.kubernetes.io/instance\":\"cert-manager\",\"app.kubernetes.io/managed-by\":\"Tiller\",\"app.kubernetes.io/name\":\"webhook\",\"helm.sh/chart\":\"webhook-v0.9.1\"},\"name\":\"cert-manager-webhook\",\"namespace\":\"cert-manager\"},\"spec\":{\"replicas\":1,\"selector\":{\"matchLabels\":{\"app\":\"webhook\",\"app.kubernetes.io/instance\":\"cert-manager\",\"app.kubernetes.io/managed-by\":\"Tiller\",\"app.kubernetes.io/name\":\"webhook\"}},\"template\":{\"metadata\":{\"annotations\":null,\"labels\":{\"app\":\"webhook\",\"app.kubernetes.io/instance\":\"cert-manager\",\"app.kubernetes.io/managed-by\":\"Tiller\",\"app.kubernetes.io/name\":\"webhook\",\"helm.sh/chart\":\"webhook-v0.9.1\"}},\"spec\":{\"containers\":[{\"args\":[\"--v=2\",\"--secure-port=6443\",\"--tls-cert-file=/certs/tls.crt\",\"--tls-private-key-file=/certs/tls.key\"],\"env\":[{\"name\":\"POD_NAMESPACE\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"metadata.namespace\"}}}],\"image\":\"quay.io/jetstack/cert-manager-webhook:v0.9.1\",\"imagePullPolicy\":\"IfNotPresent\",\"name\":\"webhook\",\"resources\":{},\"volumeMounts\":[{\"mountPath\":\"/certs\",\"name\":\"certs\"}]}],\"serviceAccountName\":\"cert-manager-webhook\",\"volumes\":[{\"name\":\"certs\",\"secret\":{\"secretName\":\"cert-manager-webhook-webhook-tls\"}}]}}}}\n"},"labels":{"app.kubernetes.io/instance":"cert-manager","app.kubernetes.io/managed-by":"Tiller","app.kubernetes.io/name":"webhook","chart":null,"helm.sh/chart":"webhook-v0.9.1","heritage":null,"release":null}},"spec":{"selector":{"matchLabels":{"app.kubernetes.io/instance":"cert-manager","app.kubernetes.io/managed-by":"Tiller","app.kubernetes.io/name":"webhook","release":null}},"template":{"metadata":{"annotations":null,"labels":{"app.kubernetes.io/instance":"cert-manager","app.kubernetes.io/managed-by":"Tiller","app.kubernetes.io/name":"webhook","helm.sh/chart":"webhook-v0.9.1","release":null}},"spec":{"$setElementOrder/containers":[{"name":"webhook"}],"containers":[{"image":"quay.io/jetstack/cert-manager-webhook:v0.9.1","name":"webhook"}]}}}}
to:
Resource: "apps/v1, Resource=deployments", GroupVersionKind: "apps/v1, Kind=Deployment"
Name: "cert-manager-webhook", Namespace: "cert-manager"
Object: &{map["apiVersion":"apps/v1" "kind":"Deployment" "metadata":map["annotations":map["deployment.kubernetes.io/revision":"1" "kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"apps/v1beta1\",\"kind\":\"Deployment\",\"metadata\":{\"annotations\":{},\"labels\":{\"app\":\"webhook\",\"chart\":\"webhook-v0.8.1\",\"heritage\":\"Tiller\",\"release\":\"cert-manager\"},\"name\":\"cert-manager-webhook\",\"namespace\":\"cert-manager\"},\"spec\":{\"replicas\":1,\"selector\":{\"matchLabels\":{\"app\":\"webhook\",\"release\":\"cert-manager\"}},\"template\":{\"metadata\":{\"annotations\":null,\"labels\":{\"app\":\"webhook\",\"release\":\"cert-manager\"}},\"spec\":{\"containers\":[{\"args\":[\"--v=2\",\"--secure-port=6443\",\"--tls-cert-file=/certs/tls.crt\",\"--tls-private-key-file=/certs/tls.key\"],\"env\":[{\"name\":\"POD_NAMESPACE\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"metadata.namespace\"}}}],\"image\":\"quay.io/jetstack/cert-manager-webhook:v0.8.1\",\"imagePullPolicy\":\"IfNotPresent\",\"name\":\"webhook\",\"resources\":{},\"volumeMounts\":[{\"mountPath\":\"/certs\",\"name\":\"certs\"}]}],\"serviceAccountName\":\"cert-manager-webhook\",\"volumes\":[{\"name\":\"certs\",\"secret\":{\"secretName\":\"cert-manager-webhook-webhook-tls\"}}]}}}}\n"] "creationTimestamp":"2019-07-16T17:56:59Z" "generation":'\x01' "labels":map["app":"webhook" "chart":"webhook-v0.8.1" "heritage":"Tiller" "release":"cert-manager"] "name":"cert-manager-webhook" "namespace":"cert-manager" "resourceVersion":"9217938" "selfLink":"/apis/apps/v1/namespaces/cert-manager/deployments/cert-manager-webhook" "uid":"1bee6b5d-a7f3-11e9-bc98-0629cdf86a62"] "spec":map["progressDeadlineSeconds":'\u0258' "replicas":'\x01' "revisionHistoryLimit":'\x02' "selector":map["matchLabels":map["app":"webhook" "release":"cert-manager"]] "strategy":map["rollingUpdate":map["maxSurge":"25%" "maxUnavailable":"25%"] "type":"RollingUpdate"] "template":map["metadata":map["creationTimestamp":<nil> "labels":map["app":"webhook" "release":"cert-manager"]] "spec":map["containers":[map["args":["--v=2" "--secure-port=6443" "--tls-cert-file=/certs/tls.crt" "--tls-private-key-file=/certs/tls.key"] "env":[map["name":"POD_NAMESPACE" "valueFrom":map["fieldRef":map["apiVersion":"v1" "fieldPath":"metadata.namespace"]]]] "image":"quay.io/jetstack/cert-manager-webhook:v0.8.1" "imagePullPolicy":"IfNotPresent" "name":"webhook" "resources":map[] "terminationMessagePath":"/dev/termination-log" "terminationMessagePolicy":"File" "volumeMounts":[map["mountPath":"/certs" "name":"certs"]]]] "dnsPolicy":"ClusterFirst" "restartPolicy":"Always" "schedulerName":"default-scheduler" "securityContext":map[] "serviceAccount":"cert-manager-webhook" "serviceAccountName":"cert-manager-webhook" "terminationGracePeriodSeconds":'\x1e' "volumes":[map["name":"certs" "secret":map["defaultMode":'\u01a4' "secretName":"cert-manager-webhook-webhook-tls"]]]]]] "status":map["availableReplicas":'\x01' "conditions":[map["lastTransitionTime":"2019-07-16T17:56:59Z" "lastUpdateTime":"2019-07-16T17:59:20Z" "message":"ReplicaSet \"cert-manager-webhook-645c7c4f5f\" has successfully progressed." "reason":"NewReplicaSetAvailable" "status":"True" "type":"Progressing"] map["lastTransitionTime":"2019-08-06T08:03:06Z" "lastUpdateTime":"2019-08-06T08:03:06Z" "message":"Deployment has minimum availability." "reason":"MinimumReplicasAvailable" "status":"True" "type":"Available"]] "observedGeneration":'\x01' "readyReplicas":'\x01' "replicas":'\x01' "updatedReplicas":'\x01']]}
for: "https://github.com/jetstack/cert-manager/releases/download/v0.9.1/cert-manager.yaml": Deployment.apps "cert-manager-webhook" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app.kubernetes.io/name":"webhook", "app":"webhook", "app.kubernetes.io/instance":"cert-manager", "app.kubernetes.io/managed-by":"Tiller"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable
Error from server (Invalid): error when applying patch:
{"metadata":{"annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"apps/v1\",\"kind\":\"Deployment\",\"metadata\":{\"annotations\":{},\"labels\":{\"app\":\"cert-manager\",\"app.kubernetes.io/instance\":\"cert-manager\",\"app.kubernetes.io/managed-by\":\"Tiller\",\"app.kubernetes.io/name\":\"cert-manager\",\"helm.sh/chart\":\"cert-manager-v0.9.1\"},\"name\":\"cert-manager\",\"namespace\":\"cert-manager\"},\"spec\":{\"replicas\":1,\"selector\":{\"matchLabels\":{\"app\":\"cert-manager\",\"app.kubernetes.io/instance\":\"cert-manager\",\"app.kubernetes.io/managed-by\":\"Tiller\",\"app.kubernetes.io/name\":\"cert-manager\"}},\"template\":{\"metadata\":{\"annotations\":{\"prometheus.io/path\":\"/metrics\",\"prometheus.io/port\":\"9402\",\"prometheus.io/scrape\":\"true\"},\"labels\":{\"app\":\"cert-manager\",\"app.kubernetes.io/instance\":\"cert-manager\",\"app.kubernetes.io/managed-by\":\"Tiller\",\"app.kubernetes.io/name\":\"cert-manager\",\"helm.sh/chart\":\"cert-manager-v0.9.1\"}},\"spec\":{\"containers\":[{\"args\":[\"--v=2\",\"--cluster-resource-namespace=$(POD_NAMESPACE)\",\"--leader-election-namespace=$(POD_NAMESPACE)\"],\"env\":[{\"name\":\"POD_NAMESPACE\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"metadata.namespace\"}}}],\"image\":\"quay.io/jetstack/cert-manager-controller:v0.9.1\",\"imagePullPolicy\":\"IfNotPresent\",\"name\":\"cert-manager\",\"ports\":[{\"containerPort\":9402}],\"resources\":{\"requests\":{\"cpu\":\"10m\",\"memory\":\"32Mi\"}}}],\"serviceAccountName\":\"cert-manager\"}}}}\n"},"labels":{"app.kubernetes.io/instance":"cert-manager","app.kubernetes.io/managed-by":"Tiller","app.kubernetes.io/name":"cert-manager","chart":null,"helm.sh/chart":"cert-manager-v0.9.1","heritage":null,"release":null}},"spec":{"selector":{"matchLabels":{"app.kubernetes.io/instance":"cert-manager","app.kubernetes.io/managed-by":"Tiller","app.kubernetes.io/name":"cert-manager","release":null}},"template":{"metadata":{"labels":{"app.kubernetes.io/instance":"cert-manager","app.kubernetes.io/managed-by":"Tiller","app.kubernetes.io/name":"cert-manager","helm.sh/chart":"cert-manager-v0.9.1","release":null}},"spec":{"$setElementOrder/containers":[{"name":"cert-manager"}],"containers":[{"image":"quay.io/jetstack/cert-manager-controller:v0.9.1","name":"cert-manager"}]}}}}
to:
Resource: "apps/v1, Resource=deployments", GroupVersionKind: "apps/v1, Kind=Deployment"
Name: "cert-manager", Namespace: "cert-manager"
Object: &{map["apiVersion":"apps/v1" "kind":"Deployment" "metadata":map["annotations":map["deployment.kubernetes.io/revision":"1" "kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"apps/v1beta1\",\"kind\":\"Deployment\",\"metadata\":{\"annotations\":{},\"labels\":{\"app\":\"cert-manager\",\"chart\":\"cert-manager-v0.8.1\",\"heritage\":\"Tiller\",\"release\":\"cert-manager\"},\"name\":\"cert-manager\",\"namespace\":\"cert-manager\"},\"spec\":{\"replicas\":1,\"selector\":{\"matchLabels\":{\"app\":\"cert-manager\",\"release\":\"cert-manager\"}},\"template\":{\"metadata\":{\"annotations\":{\"prometheus.io/path\":\"/metrics\",\"prometheus.io/port\":\"9402\",\"prometheus.io/scrape\":\"true\"},\"labels\":{\"app\":\"cert-manager\",\"release\":\"cert-manager\"}},\"spec\":{\"containers\":[{\"args\":[\"--v=2\",\"--cluster-resource-namespace=$(POD_NAMESPACE)\",\"--leader-election-namespace=$(POD_NAMESPACE)\"],\"env\":[{\"name\":\"POD_NAMESPACE\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"metadata.namespace\"}}}],\"image\":\"quay.io/jetstack/cert-manager-controller:v0.8.1\",\"imagePullPolicy\":\"IfNotPresent\",\"name\":\"cert-manager\",\"ports\":[{\"containerPort\":9402}],\"resources\":{\"requests\":{\"cpu\":\"10m\",\"memory\":\"32Mi\"}}}],\"serviceAccountName\":\"cert-manager\"}}}}\n"] "creationTimestamp":"2019-07-16T17:57:00Z" "generation":'\x01' "labels":map["app":"cert-manager" "chart":"cert-manager-v0.8.1" "heritage":"Tiller" "release":"cert-manager"] "name":"cert-manager" "namespace":"cert-manager" "resourceVersion":"9217847" "selfLink":"/apis/apps/v1/namespaces/cert-manager/deployments/cert-manager" "uid":"1c1c9333-a7f3-11e9-bc98-0629cdf86a62"] "spec":map["progressDeadlineSeconds":'\u0258' "replicas":'\x01' "revisionHistoryLimit":'\x02' "selector":map["matchLabels":map["app":"cert-manager" "release":"cert-manager"]] "strategy":map["rollingUpdate":map["maxSurge":"25%" "maxUnavailable":"25%"] "type":"RollingUpdate"] "template":map["metadata":map["annotations":map["prometheus.io/path":"/metrics" "prometheus.io/port":"9402" "prometheus.io/scrape":"true"] "creationTimestamp":<nil> "labels":map["app":"cert-manager" "release":"cert-manager"]] "spec":map["containers":[map["args":["--v=2" "--cluster-resource-namespace=$(POD_NAMESPACE)" "--leader-election-namespace=$(POD_NAMESPACE)"] "env":[map["name":"POD_NAMESPACE" "valueFrom":map["fieldRef":map["apiVersion":"v1" "fieldPath":"metadata.namespace"]]]] "image":"quay.io/jetstack/cert-manager-controller:v0.8.1" "imagePullPolicy":"IfNotPresent" "name":"cert-manager" "ports":[map["containerPort":'\u24ba' "protocol":"TCP"]] "resources":map["requests":map["cpu":"10m" "memory":"32Mi"]] "terminationMessagePath":"/dev/termination-log" "terminationMessagePolicy":"File"]] "dnsPolicy":"ClusterFirst" "restartPolicy":"Always" "schedulerName":"default-scheduler" "securityContext":map[] "serviceAccount":"cert-manager" "serviceAccountName":"cert-manager" "terminationGracePeriodSeconds":'\x1e']]] "status":map["availableReplicas":'\x01' "conditions":[map["lastTransitionTime":"2019-07-16T17:57:00Z" "lastUpdateTime":"2019-07-16T17:57:01Z" "message":"ReplicaSet \"cert-manager-8d478bb45\" has successfully progressed." "reason":"NewReplicaSetAvailable" "status":"True" "type":"Progressing"] map["lastTransitionTime":"2019-08-06T08:02:41Z" "lastUpdateTime":"2019-08-06T08:02:41Z" "message":"Deployment has minimum availability." "reason":"MinimumReplicasAvailable" "status":"True" "type":"Available"]] "observedGeneration":'\x01' "readyReplicas":'\x01' "replicas":'\x01' "updatedReplicas":'\x01']]}
for: "https://github.com/jetstack/cert-manager/releases/download/v0.9.1/cert-manager.yaml": Deployment.apps "cert-manager" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app.kubernetes.io/name":"cert-manager", "app":"cert-manager", "app.kubernetes.io/instance":"cert-manager", "app.kubernetes.io/managed-by":"Tiller"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable

k version                                                                                                                            

Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.2", GitCommit:"f6278300bebbb750328ac16ee6dd3aa7d3549568", GitTreeState:"clean", BuildDate:"2019-08-05T16:54:35Z", GoVersion:"go1.12.7", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.8", GitCommit:"a89f8c11a5f4f132503edbc4918c98518fd504e3", GitTreeState:"clean", BuildDate:"2019-04-23T04:41:47Z", GoVersion:"go1.10.8", Compiler:"gc", Platform:"linux/amd64"}

AWS/kops.

I am unsure how to proceed since the upgrade breaks half way.

Many thanks for any info,
Rich

[vreon]

Same problem as #1927, only without Helm. I hit this too, going from 0.7.2 to 0.9.1.

In my case, I needed to delete the existing Deployments and then reapply.

[richstokes]

Thanks @vreon, I ran kubectl replace --force -f https://github.com/jetstack/cert-manager/releases/download/v0.9.1/cert-manager.yaml --validate=false based on the suggestions in #1927

I then also had to re-apply my ClusterIssuer (as I guess it got deleted), and AWS secret for cert-manager (I use the DNS method so need this to tap into Route53).

After doing the above everything seems to be working.

I think the upgrade docs (or the pre-release testing process) needs updating as upgrades are not working as described.

[JoshVanL]

Hey guys, thanks for bringing this up. There is a PR open which should help clarify the problem

https://github.com/jetstack/cert-manager/pull/1999

[richstokes]

For anyone stumbling upon this, you might not want to follow my commands exactly ^^

The replace/force stuff actually deleted the certificate resource records. The secrets still exist, so the sites are working atm, but not sure if cert-manager can/will recover from this automatically.

As a result I am getting a lot of "unable to fetch certificate that owns the secret" in the logs.

The work around seems to be to edit the secret name in the ingress resource. This then triggers cert-manager to request a new cert.

Is there a better way to have cert-manager rebuild the missing certificate records?

[richstokes]

I ended up creating a script to delete and re-add the ingress records. This then triggered cert-manager to do its thing and create the certificate records correctly.

https://github.com/richstokes/k8s-scripts/blob/master/refresh-ingress/refresh-ing.sh