Cert-manager: Validation stops certificate creation

[justinsb]

While trying to use cert-manager 0.6.1 with the no-webhook configuration, on a GKE cluster running 1.11.6, I encountered this error in the cert-manager logs status.certificate in body must be of type string: "null". Context with minor redaction...:

cert-manager-7476cc944f-7m9cg cert-manager I0216 04:11:06.017330       1 controller.go:145] certificates controller: syncing item 'artifactserver/wildcard-example-com'
cert-manager-7476cc944f-7m9cg cert-manager I0216 04:11:06.024276       1 sync.go:354] Error issuing certificate for artifactserver/wildcard-example-com: Order.certmanager.k8s.io "wildcard-example-com-1312178538" is invalid: []: Invalid value: map[string]interface {}{"kind":"Order", "apiVersion":"certmanager.k8s.io/v1alpha1", "metadata":map[string]interface {}{"uid":"e18e1cad-31a0-11e9-a817-42010a80006c", "selfLink":"", "name":"wildcard-example-com-1312178538", "namespace":"artifactserver", "creationTimestamp":"2019-02-16T04:11:06Z", "labels":map[string]interface {}{"acme.cert-manager.io/certificate-name":"wildcard-example-com"}, "ownerReferences":[]interface {}{map[string]interface {}{"name":"wildcard-example-com", "uid":"5c1d5dac-31a0-11e9-a817-42010a80006c", "controller":true, "blockOwnerDeletion":true, "apiVersion":"certmanager.k8s.io/v1alpha1", "kind":"Certificate"}}, "generation":1}, "spec":map[string]interface {}{"dnsNames":[]interface {}{"*.example.com"}, "config":[]interface {}{map[string]interface {}{"domains":[]interface {}{"*.example.com"}, "dns01":map[string]interface {}{"provider":"default"}}}, "csr":"...", "issuerRef":map[string]interface {}{"name":"letsencrypt"}}, "status":map[string]interface {}{"state":"", "reason":"", "url":"", "finalizeURL":"", "certificate":interface {}(nil)}}: validation failure list:
cert-manager-7476cc944f-7m9cg cert-manager status.certificate in body must be of type string: "null"

By removing the validation block from the Order CRD entirely I was able to proceed, and a certificate was issued (yay). But it looks like status.certificate is being treated as a required field.

[munnerz]

It looks like you've applied the CRD manifests from the master branch instead of the release-0.6 branch, hence why you're getting this error.

We've recently merged in a validation schema to master that is due to be released as part of v0.7, however it's not a part of v0.6 as we sort of expected there to be a few edge cases like this 😄

Can you confirm that your configuration works okay when you use the versioned CRD manifests instead?

I'm going to label this as a bug, as it is something that needs fixing for v0.7 😄

/kind bug
/milestone v0.7
/priority important-soon
/area api

[munnerz]

After digging in some more, this seems like it isn't a bug and is simply a case of you using the 'master' version of the CRDs with the v0.6.x release of cert-manager.

As part of the PR that introduced the CRD validation schema, we added omitempty to a number of fields that were missing it. By using the older version of CM, that does not include these omitempty lines, you're now seeing 'null' being set as the 'Certificate' value, which is indeed invalid.

You should either switch to use the :canary image tags for CM, or otherwise use the correct version of the CRD manifests along with the correct version of cert-manager :smile:

Hope that helps/explains what's going on!

/close

[jetstack-bot]

@munnerz: Closing this issue.

In response to this:

After digging in some more, this seems like it isn't a bug and is simply a case of you using the 'master' version of the CRDs with the v0.6.x release of cert-manager.

As part of the PR that introduced the CRD validation schema, we added omitempty to a number of fields that were missing it. By using the older version of CM, that does not include these omitempty lines, you're now seeing 'null' being set as the 'Certificate' value, which is indeed invalid.

You should either switch to use the :canary image tags for CM, or otherwise use the correct version of the CRD manifests along with the correct version of cert-manager :smile:

Hope that helps/explains what's going on!

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[Sahasrara]

Hey @munnerz, I was following these instructions:
https://docs.cert-manager.io/en/latest/getting-started/install.html

Then I tried to create a certificate:

apiVersion: certmanager.k8s.io/v1alpha1
kind: Certificate
metadata:
  name: nginx-gateway-certificate
  namespace: default
spec:
  secretName: nginx-gateway-certificate
  issuerRef:
    name: letsencrypt-prod
    kind: ClusterIssuer
  dnsNames:
    - <redacted>
  acme:
    config:
      - http01:
          ingressClass: nginx
        domains:
          - <redacted>

And I'm getting the same error:

E0416 18:45:30.587251       1 controller.go:147] certificates controller: Re-queuing item "default/nginx-gateway-certificate" due to error processing: Order.certmanager.k8s.io "nginx-gateway-certificate-3277600406" is invalid: []: Invalid value: map[string]interface {}{"apiVersion":"certmanager.k8s.io/v1alpha1", "metadata":map[string]interface {}{"name":"nginx-gateway-certificate-3277600406", "namespace":"default", "creationTimestamp":"2019-04-16T18:45:30Z", "labels":map[string]interface {}{"acme.cert-manager.io/certificate-name":"nginx-gateway-certificate"}, "ownerReferences":[]interface {}{map[string]interface {}{"apiVersion":"certmanager.k8s.io/v1alpha1", "kind":"Certificate", "name":"nginx-gateway-certificate", "uid":"c5f8c5f7-6077-11e9-9b0e-520d296f8497", "controller":true, "blockOwnerDeletion":true}}, "generation":1, "uid":"cf3f3183-6077-11e9-9b0e-520d296f8497"}, "spec":map[string]interface {}{"csr":"MIICvzCCAacCAQAwPzEVMBMGA1UEChMMY2VydC1tYW5hZ2VyMSYwJAYDVQQDEx1nYXRld2F5LmFkcy5henVyZS5teXRla3NpLm5ldDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMlY+3noZNoY5RcQ6pAm6kmiXxgnsvoGpHsoHdueS7/42hFMP5YsuHSFd3RulNgh0e8kTqFPKEPF8ZSvwU3cL+GZ3CqJD0zYoIVg41JBMyZyyrKb5ClWLjPm79/BnKVgOfODvBXx8OiiqeI3gUaJC5CethoS2yMNzrHWZgF6BfJ0Zcoa9vt6xypJNTvOTHpqL35szFd+0b306Dmi1qeoyNp96wUjj8amlqZWgmlNXaa2fRgEZyf+hQcbM/rdbQRIxyyEr3jSY4klC+AoVAWp6uNJavATmTqLsE9G68jIVHz6pa0/Dajsmb5uCutwp/GLxaMjD7xS6QvGnVzmkb11CS0CAwEAAaA7MDkGCSqGSIb3DQEJDjEsMCowKAYDVR0RBCEwH4IdZ2F0ZXdheS5hZHMuYXp1cmUubXl0ZWtzaS5uZXQwDQYJKoZIhvcNAQELBQADggEBAA6OmbYKWAjXKSSR6up04DtWIwSwF18fy3TKcA4NB2MRQtgl2ElGwIXEmBDRWycDhPPtGLLq2zb74rL3isFRa4v61tpGj3PNqeDFe+d2XZ6NqtpEGGzZjXmu9cM6TfzVwS5JqBZNOrLJK1QEROSrzfDKyOwFZAPilWHGmYzL6XOUDPasRiySEzKY1fYOWQfHnxZUaNxULMMEcMQL1RIUQow47L3IbKV8hploJZ9XLJkaHEF0/TY3iWBlUIvZITs7SSadyapScXGDvvxv3w5FhXNFTNR8XR8yO3WYjmrfcvUG1xrQo6tZmtJcrnpGexOLfR0ZLxxwMfKv9FiNZBztnKE=", "issuerRef":map[string]interface {}{"kind":"ClusterIssuer", "name":"letsencrypt-prod"}, "dnsNames":[]interface {}{"<redacted>"}, "config":[]interface {}{map[string]interface {}{"domains":[]interface {}{"<redacted>"}, "http01":map[string]interface {}{"ingress":"", "ingressClass":"nginx"}}}}, "status":map[string]interface {}{"url":"", "finalizeURL":"", "certificate":interface {}(nil), "state":"", "reason":""}, "kind":"Order"}: validation failure list:
status.certificate in body must be of type string: "null"

[Sahasrara]

@munnerz ^^

[Sahasrara]

@munnerz Just found out that the reason this isn't working is the installation documentation is wrong.

It says:

# Install the cert-manager Helm chart
helm install \
  --name cert-manager \
  --namespace cert-manager \
  --version v0.7.2 \
  jetstack/cert-manager

but --version is for the chart, not for the image.

@posterity