Cert-manager: 'kubectl get certificates' no longer returns cert status

[snobu]

Most probably this isn't a bug but a change in behavior. Before v0.7.0, a kubectl get certificates would list the certs with a True/False status field.

Looks like v0.7.0 no longer does this:

$ kubectl get certificates -o wide
NAME
canary-contour-tls
test-contour-tls

The field is definitely still there just not returned by default,

$ kubectl get certificates canary-contour-tls -o yaml
...
status:
  conditions:
  - lastTransitionTime: "2019-03-13T17:58:51Z"
    message: Certificate is up to date and has not expired
    reason: Ready
    status: "True"
    type: Ready
  notAfter: "2019-06-11T16:58:50Z"

Is my installation broken or is there now a different way to get cert ready status?

[Evesy]

Would you mind sharing the output of kubectl get customresourcedefinitions certificates.certmanager.k8s.io -o yaml?

From what I can see there shouldn't have been any behaviour change in how those fields are displayed

[snobu]

I've made quite a few changes to this cluster lately so it could very well be something i screwed up and not cert-manager at all.

Here's the output.

$ kubectl get customresourcedefinitions certificates.certmanager.k8s.io -o yaml

apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"apiextensions.k8s.io/v1beta1","kind":"CustomResourceDefinition","metadata":{"annotations":{},"creationTimestamp":null,"labels":{"controller-tools.k8s.io":"1.0"},"name":"certificates.certmanager.k8s.io"},"spec":{"additionalPrinterColumns":[{"JSONPath":".status.conditions[?(@.type==\\\"Ready\\\")].status","name":"Ready","type":"string"},{"JSONPath":".spec.secretName","name":"Secret","type":"string"},{"JSONPath":".spec.issuerRef.name","name":"Issuer","priority":1,"type":"string"},{"JSONPath":".status.conditions[?(@.type==\\\"Ready\\\")].message","name":"Status","priority":1,"type":"string"},{"JSONPath":".metadata.creationTimestamp","description":"CreationTimestamp is a timestamp representing the server time when this object was created. It is not guaranteed to be set in happens-before order across separate operations. Clients may not set this value. It is represented in RFC3339 form and is in UTC.","name":"Age","type":"date"}],"group":"certmanager.k8s.io","names":{"kind":"Certificate","plural":"certificates","shortNames":["cert","certs"]},"scope":"Namespaced","validation":{"openAPIV3Schema":{"properties":{"apiVersion":{"description":"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources","type":"string"},"kind":{"description":"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds","type":"string"},"metadata":{"type":"object"},"spec":{"properties":{"acme":{"description":"ACME contains configuration specific to ACME Certificates. Notably, this contains details on how the domain names listed on this Certificate resource should be 'solved', i.e. mapping HTTP01 and DNS01 providers to DNS names.","properties":{"config":{"items":{"properties":{"domains":{"description":"Domains is the list of domains that this SolverConfig applies to.","items":{"type":"string"},"type":"array"}},"required":["domains"],"type":"object"},"type":"array"}},"required":["config"],"type":"object"},"commonName":{"description":"CommonName is a common name to be used on the Certificate","type":"string"},"dnsNames":{"description":"DNSNames is a list of subject alt names to be used on the Certificate","items":{"type":"string"},"type":"array"},"duration":{"description":"Certificate default Duration","type":"string"},"ipAddresses":{"description":"IPAddresses is a list of IP addresses to be used on the Certificate","items":{"type":"string"},"type":"array"},"isCA":{"description":"IsCA will mark this Certificate as valid for signing. This implies that the 'signing' usage is set","type":"boolean"},"issuerRef":{"description":"IssuerRef is a reference to the issuer for this certificate. If the 'kind' field is not set, or set to 'Issuer', an Issuer resource with the given name in the same namespace as the Certificate will be used. If the 'kind' field is set to 'ClusterIssuer', a ClusterIssuer with the provided name will be used. The 'name' field in this stanza is required at all times.","properties":{"kind":{"type":"string"},"name":{"type":"string"}},"required":["name"],"type":"object"},"keyAlgorithm":{"description":"KeyAlgorithm is the private key algorithm of the corresponding private key for this certificate. If provided, allowed values are either \"rsa\" or \"ecdsa\" If KeyAlgorithm is specified and KeySize is not provided, key size of 256 will be used for \"ecdsa\" key algorithm and key size of 2048 will be used for \"rsa\" key algorithm.","enum":["rsa","ecdsa"],"type":"string"},"keySize":{"description":"KeySize is the key bit size of the corresponding private key for this certificate. If provided, value must be between 2048 and 8192 inclusive when KeyAlgorithm is empty or is set to \"rsa\", and value must be one of (256, 384, 521) when KeyAlgorithm is set to \"ecdsa\".","format":"int64","type":"integer"},"organization":{"description":"Organization is the organization to be used on the Certificate","items":{"type":"string"},"type":"array"},"renewBefore":{"description":"Certificate renew before expiration duration","type":"string"},"secretName":{"description":"SecretName is the name of the secret resource to store this secret in","type":"string"}},"required":["secretName","issuerRef"],"type":"object"},"status":{"properties":{"conditions":{"items":{"properties":{"lastTransitionTime":{"description":"LastTransitionTime is the timestamp corresponding to the last status change of this condition.","format":"date-time","type":"string"},"message":{"description":"Message is a human readable description of the details of the last transition, complementing reason.","type":"string"},"reason":{"description":"Reason is a brief machine readable explanation for the condition's last transition.","type":"string"},"status":{"description":"Status of the condition, one of ('True', 'False', 'Unknown').","enum":["True","False","Unknown"],"type":"string"},"type":{"description":"Type of the condition, currently ('Ready').","type":"string"}},"required":["type","status","lastTransitionTime","reason","message"],"type":"object"},"type":"array"},"lastFailureTime":{"format":"date-time","type":"string"},"notAfter":{"description":"The expiration time of the certificate stored in the secret named by this resource in spec.secretName.","format":"date-time","type":"string"}},"type":"object"}}}},"version":"v1alpha1"},"status":{"acceptedNames":{"kind":"","plural":""},"conditions":[],"storedVersions":[]}}
  creationTimestamp: "2019-03-01T20:32:39Z"
  generation: 2
  labels:
    controller-tools.k8s.io: "1.0"
  name: certificates.certmanager.k8s.io
  resourceVersion: "2063053"
  selfLink: /apis/apiextensions.k8s.io/v1beta1/customresourcedefinitions/certificates.certmanager.k8s.io
  uid: 281b954e-3c61-11e9-8060-a65061dfcfa1
spec:
  additionalPrinterColumns:
  - JSONPath: .status.conditions[?(@.type==\"Ready\")].status
    name: Ready
    type: string
  - JSONPath: .spec.secretName
    name: Secret
    type: string
  - JSONPath: .spec.issuerRef.name
    name: Issuer
    priority: 1
    type: string
  - JSONPath: .status.conditions[?(@.type==\"Ready\")].message
    name: Status
    priority: 1
    type: string
  - JSONPath: .metadata.creationTimestamp
    description: CreationTimestamp is a timestamp representing the server time when
      this object was created. It is not guaranteed to be set in happens-before order
      across separate operations. Clients may not set this value. It is represented
      in RFC3339 form and is in UTC.
    name: Age
    type: date
  group: certmanager.k8s.io
  names:
    kind: Certificate
    listKind: CertificateList
    plural: certificates
    shortNames:
    - cert
    - certs
    singular: certificate
  scope: Namespaced
  validation:
    openAPIV3Schema:
      properties:
        apiVersion:
          description: 'APIVersion defines the versioned schema of this representation
            of an object. Servers should convert recognized schemas to the latest
            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources'
          type: string
        kind:
          description: 'Kind is a string value representing the REST resource this
            object represents. Servers may infer this from the endpoint the client
            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds'
          type: string
        metadata:
          type: object
        spec:
          properties:
            acme:
              description: ACME contains configuration specific to ACME Certificates.
                Notably, this contains details on how the domain names listed on this
                Certificate resource should be 'solved', i.e. mapping HTTP01 and DNS01
                providers to DNS names.
              properties:
                config:
                  items:
                    properties:
                      domains:
                        description: Domains is the list of domains that this SolverConfig
                          applies to.
                        items:
                          type: string
                        type: array
                    required:
                    - domains
                    type: object
                  type: array
              required:
              - config
              type: object
            commonName:
              description: CommonName is a common name to be used on the Certificate
              type: string
            dnsNames:
              description: DNSNames is a list of subject alt names to be used on the
                Certificate
              items:
                type: string
              type: array
            duration:
              description: Certificate default Duration
              type: string
            ipAddresses:
              description: IPAddresses is a list of IP addresses to be used on the
                Certificate
              items:
                type: string
              type: array
            isCA:
              description: IsCA will mark this Certificate as valid for signing. This
                implies that the 'signing' usage is set
              type: boolean
            issuerRef:
              description: IssuerRef is a reference to the issuer for this certificate.
                If the 'kind' field is not set, or set to 'Issuer', an Issuer resource
                with the given name in the same namespace as the Certificate will
                be used. If the 'kind' field is set to 'ClusterIssuer', a ClusterIssuer
                with the provided name will be used. The 'name' field in this stanza
                is required at all times.
              properties:
                kind:
                  type: string
                name:
                  type: string
              required:
              - name
              type: object
            keyAlgorithm:
              description: KeyAlgorithm is the private key algorithm of the corresponding
                private key for this certificate. If provided, allowed values are
                either "rsa" or "ecdsa" If KeyAlgorithm is specified and KeySize is
                not provided, key size of 256 will be used for "ecdsa" key algorithm
                and key size of 2048 will be used for "rsa" key algorithm.
              enum:
              - rsa
              - ecdsa
              type: string
            keySize:
              description: KeySize is the key bit size of the corresponding private
                key for this certificate. If provided, value must be between 2048
                and 8192 inclusive when KeyAlgorithm is empty or is set to "rsa",
                and value must be one of (256, 384, 521) when KeyAlgorithm is set
                to "ecdsa".
              format: int64
              type: integer
            organization:
              description: Organization is the organization to be used on the Certificate
              items:
                type: string
              type: array
            renewBefore:
              description: Certificate renew before expiration duration
              type: string
            secretName:
              description: SecretName is the name of the secret resource to store
                this secret in
              type: string
          required:
          - secretName
          - issuerRef
          type: object
        status:
          properties:
            conditions:
              items:
                properties:
                  lastTransitionTime:
                    description: LastTransitionTime is the timestamp corresponding
                      to the last status change of this condition.
                    format: date-time
                    type: string
                  message:
                    description: Message is a human readable description of the details
                      of the last transition, complementing reason.
                    type: string
                  reason:
                    description: Reason is a brief machine readable explanation for
                      the condition's last transition.
                    type: string
                  status:
                    description: Status of the condition, one of ('True', 'False',
                      'Unknown').
                    enum:
                    - "True"
                    - "False"
                    - Unknown
                    type: string
                  type:
                    description: Type of the condition, currently ('Ready').
                    type: string
                required:
                - type
                - status
                - lastTransitionTime
                - reason
                - message
                type: object
              type: array
            lastFailureTime:
              format: date-time
              type: string
            notAfter:
              description: The expiration time of the certificate stored in the secret
                named by this resource in spec.secretName.
              format: date-time
              type: string
          type: object
  version: v1alpha1
  versions:
  - name: v1alpha1
    served: true
    storage: true
status:
  acceptedNames:
    kind: Certificate
    listKind: CertificateList
    plural: certificates
    shortNames:
    - cert
    - certs
    singular: certificate
  conditions:
  - lastTransitionTime: "2019-03-01T20:32:39Z"
    message: no conflicts found
    reason: NoConflicts
    status: "True"
    type: NamesAccepted
  - lastTransitionTime: null
    message: the initial names have been accepted
    reason: InitialNamesAccepted
    status: "True"
    type: Established
  storedVersions:
  - v1alpha1

[lsowen]

I am also having the same issue, after upgrading from v0.6.2 (where it did work properly) to v0.7.0.

Tracked it down to this change: https://github.com/jetstack/cert-manager/commit/e29c31f9db5b134e4ae4eef0c1f0ef147faa4a46#diff-e68711314053342f65dbf1e77774b889L9

When I manually edit the CRD (edit crd certificates.certmanager.k8s.io) and remove the backslashes, it starts working for me again.

[snobu]

That seems to be it, removing the escape chars does the trick:

$ kubectl get certificates
NAME                                READY   SECRET               AGE
canary-contour-tls                  True    canary-contour-tls   4d
echo-contour-tls                    True    echo-contour-tls     4d

$ helm ls | grep cert
cert-manager          DEPLOYED        cert-manager-v0.7.0 

[Aliabbask08]

list out all csr use:
kubectl get csr -o wide