Cert-manager: Certificate with ready status false

Created on 4 Dec 2019  路  8Comments  路  Source: jetstack/cert-manager

Describe the bug:
Deleting the secret with have the "TLS certificate", the resource Certificate gets the state "ready false", the cert-manager log is repeatedly printed CertificateRequest contains a valid certificate for issuance. Issuing certificate ...

I1203 23:44:29.990254 1 sync.go:442] cert-manager/controller/certificates "level"=0 "msg"="CertificateRequest is in a Ready state, issuing certificate..." "related_resource_kind"="CertificateRequest" "related_resource_name"="letsencrypt-certificate-servicex-3038630818" "related_resource_namespace"="xuwevkvketfixgfunx" "resource_kind"="Certificate" "resource_name"="letsencrypt-certificate-servicex" "resource_namespace"="xuwevkvketfixgfunx" I1203 23:44:29.990284 1 sync.go:445] cert-manager/controller/certificates "level"=0 "msg"="decoding certificate data" "related_resource_kind"="CertificateRequest" "related_resource_name"="letsencrypt-certificate-servicex-3038630818" "related_resource_namespace"="xuwevkvketfixgfunx" "resource_kind"="Certificate" "resource_name"="letsencrypt-certificate-servicex" "resource_namespace"="xuwevkvketfixgfunx" I1203 23:44:29.990500 1 sync.go:453] cert-manager/controller/certificates "level"=0 "msg"="checking if certificate stored on CertificateRequest is up to date" "related_resource_kind"="CertificateRequest" "related_resource_name"="letsencrypt-certificate-servicex-3038630818" "related_resource_namespace"="xuwevkvketfixgfunx" "resource_kind"="Certificate" "resource_name"="letsencrypt-certificate-servicex" "resource_namespace"="xuwevkvketfixgfunx" I1203 23:44:29.990534 1 sync.go:466] cert-manager/controller/certificates "level"=0 "msg"="CertificateRequest contains a valid certificate for issuance. Issuing certificate..." "related_resource_kind"="CertificateRequest" "related_resource_name"="letsencrypt-certificate-servicex-3038630818" "related_resource_namespace"="xuwevkvketfixgfunx" "resource_kind"="Certificate" "resource_name"="letsencrypt-certificate-servicex" "resource_namespace"="xuwevkvketfixgfunx" I1203 23:44:30.189388 1 controller.go:135] cert-manager/controller/certificates "level"=0 "msg"="finished processing work item" "key"="xuwevkvketfixgfunx/letsencrypt-certificate-servicex" I1203 23:44:30.189437 1 controller.go:129] cert-manager/controller/certificates "level"=0 "msg"="syncing item" "key"="xuwevkvketfixgfunx/letsencrypt-certificate-servicex" I1203 23:44:30.190060 1 sync.go:379] cert-manager/controller/certificates "level"=0 "msg"="validating existing CSR data" "related_resource_kind"="CertificateRequest" "related_resource_name"="letsencrypt-certificate-servicex-3038630818" "related_resource_namespace"="xuwevkvketfixgfunx" "resource_kind"="Certificate" "resource_name"="letsencrypt-certificate-servicex" "resource_namespace"="xuwevkvketfixgfunx" I1203 23:44:30.190176 1 sync.go:442] cert-manager/controller/certificates "level"=0 "msg"="CertificateRequest is in a Ready state, issuing certificate..." "related_resource_kind"="CertificateRequest" "related_resource_name"="letsencrypt-certificate-servicex-3038630818" "related_resource_namespace"="xuwevkvketfixgfunx" "resource_kind"="Certificate" "resource_name"="letsencrypt-certificate-servicex" "resource_namespace"="xuwevkvketfixgfunx" I1203 23:44:30.190218 1 sync.go:445] cert-manager/controller/certificates "level"=0 "msg"="decoding certificate data" "related_resource_kind"="CertificateRequest" "related_resource_name"="letsencrypt-certificate-servicex-3038630818" "related_resource_namespace"="xuwevkvketfixgfunx" "resource_kind"="Certificate" "resource_name"="letsencrypt-certificate-servicex" "resource_namespace"="xuwevkvketfixgfunx" I1203 23:44:30.190494 1 sync.go:453] cert-manager/controller/certificates "level"=0 "msg"="checking if certificate stored on CertificateRequest is up to date" "related_resource_kind"="CertificateRequest" "related_resource_name"="letsencrypt-certificate-servicex-3038630818" "related_resource_namespace"="xuwevkvketfixgfunx" "resource_kind"="Certificate" "resource_name"="letsencrypt-certificate-servicex" "resource_namespace"="xuwevkvketfixgfunx" I1203 23:44:30.190532 1 sync.go:466] cert-manager/controller/certificates "level"=0 "msg"="CertificateRequest contains a valid certificate for issuance. Issuing certificate..." "related_resource_kind"="CertificateRequest" "related_resource_name"="letsencrypt-certificate-servicex-3038630818" "related_resource_namespace"="xuwevkvketfixgfunx" "resource_kind"="Certificate" "resource_name"="letsencrypt-certificate-servicex" "resource_namespace"="xuwevkvketfixgfunx" I1203 23:44:30.389043 1 controller.go:135] cert-manager/controller/certificates "level"=0 "msg"="finished processing work item" "key"="xuwevkvketfixgfunx/letsencrypt-certificate-servicex" I1203 23:44:30.389143 1 controller.go:129] cert-manager/controller/certificates "level"=0 "msg"="syncing item" "key"="xuwevkvketfixgfunx/letsencrypt-certificate-servicex" I1203 23:44:30.389847 1 sync.go:379] cert-manager/controller/certificates "level"=0 "msg"="validating existing CSR data" "related_resource_kind"="CertificateRequest" "related_resource_name"="letsencrypt-certificate-servicex-3038630818" "related_resource_namespace"="xuwevkvketfixgfunx" "resource_kind"="Certificate" "resource_name"="letsencrypt-certificate-servicex" "resource_namespace"="xuwevkvketfixgfunx"

Expected behaviour:
The resource Certificate with state ready true after CertificateRequest has success

Steps to reproduce the bug:
kubectl delete secrets letsencrypt-certificate-servicex

kubectl describe Certificate letsencrypt-certificate-servicex
Message: Waiting for CertificateRequest "letsencrypt-certificate-servicex-3038630818" to complete
Reason: InProgress
Status: False
Type: Ready

kubectl describe -n xuwevkvketfixgfunx CertificateRequest letsencrypt-certificate-servicex-3038630818
Message: Certificate fetched from issuer successfully
Reason: Issued
Status: True
Type: Ready

Anything else we need to know?:

  • Certificate status was "Ready true" before being deleted
  • The secret was regenerated after being deleted for the first time
  • After deleting a second time, the resource Certificate gets state ready

Environment details::

  • Kubernetes version: 1.14.7-gke.23
  • Cloud-provider/provisioner: GKE
  • cert-manager version v0.11.1 and v0.12.0
  • Install method: helm

Can this behavior occur on automatic certificate renewal?

/kind bug

areacme kinbug
Source
picture ifnazar

Most helpful comment

@mvdan in this case the resource CertificateRequest succeeded, but orders and challenge were not created, no error message, only log is looped with message the CertificateRequest contains a valid certificate for issuance. Issuing certificate ...
_( the status only returns to normal when I delete the secret with the key a second time )_

delete-secret

picture ifnazar on 5 Dec 2019
👍5

All 8 comments

Have you had a look at https://github.com/jetstack/cert-manager/issues/2388? It could be the same issue.

mvdan picture mvdan on 4 Dec 2019

@mvdan in this case the resource CertificateRequest succeeded, but orders and challenge were not created, no error message, only log is looped with message the CertificateRequest contains a valid certificate for issuance. Issuing certificate ...
_( the status only returns to normal when I delete the secret with the key a second time )_

delete-secret

ifnazar picture ifnazar on 5 Dec 2019
👍5

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Send feedback to jetstack.
/lifecycle stale

retest-bot picture retest-bot on 4 Mar 2020

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.
Send feedback to jetstack.
/lifecycle rotten
/remove-lifecycle stale

retest-bot picture retest-bot on 3 Apr 2020

/remove-lifecycle rotten

sdanbury picture sdanbury on 20 Apr 2020

Still seeing this in v1.15.9 kubernetes with v0.13.0 cert-manager-controller.

sdanbury picture sdanbury on 20 Apr 2020
👍1

Thanks for the bug report.
This was fixed in master by #2539 and in https://github.com/jetstack/cert-manager/releases/tag/v0.13.1 by #2543

Please try upgrading cert-manager and report back if you still see this issue.

The repeated log message are explained by @munnerz in https://github.com/jetstack/cert-manager/pull/2539#discussion_r370035634

/area acme
/close

wallrj picture wallrj on 24 Apr 2020

@wallrj: Closing this issue.

In response to this:

Thanks for the bug report.
This was fixed in master by #2539 and in https://github.com/jetstack/cert-manager/releases/tag/v0.13.1 by #2543

Please try upgrading cert-manager and report back if you still see this issue.

The repeated log message are explained by @munnerz in https://github.com/jetstack/cert-manager/pull/2539#discussion_r370035634

/area acme
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

jetstack-bot picture jetstack-bot on 24 Apr 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

dontreboot picture dontreboot  路  3Comments
munjal-patel picture munjal-patel  路  3Comments
jbeda picture jbeda  路  4Comments
timblakely picture timblakely  路  4Comments
kragniz picture kragniz  路  4Comments