cert-manager "successfully" create certificate but no secret is actually created

[victornoel]

Bugs should be filed for issues encountered whilst operating cert-manager.
You should first attempt to resolve your issues through the community support
channels, e.g. Slack, in order to rule out individual configuration errors.
Please provide as much detail as possible.

Describe the bug:

I have setup my ingress following http://docs.cert-manager.io/en/latest/reference/ingress-shim.html with a ClusterIssues (letsencrypt staging).
cert-manager emits the following event:

Normal  CreateCertificate  4m13s                cert-manager              Successfully created Certificate "tls-cert-staging.xxxx.com"

But no secret is created and thus nginx-ingress can't find it.

Expected behaviour:

A secret must either be created or an error printed somewhere in the logs instead of a message of successful creation.

Steps to reproduce the bug:

Here is my ingress resource (as given to k8s):

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: blue-green-staging
  labels:    
    app.kubernetes.io/name: blue-green
    app.kubernetes.io/instance: bg
    helm.sh/chart: blue-green-1.0.0
    app.kubernetes.io/managed-by: Tiller
  annotations:
    certmanager.k8s.io/cluster-issuer: "letsencrypt-staging"
    kubernetes.io/ingress.class: nginx
spec:
  rules:
    - host: "staging.xxxx.com"
      http:
        paths:
          - path: /
            backend:
              serviceName: blue-green-staging
              servicePort: http
  tls:
    - secretName: "tls-cert-staging.xxxx.com"
      hosts:
        - "staging.xxxx.com"

Describe on this resource gives me:

Name:             blue-green-staging
Namespace:        default
Address:          a.b.c.d
Default backend:  default-http-backend:80 (<none>)
TLS:
  tls-cert-staging.xxxx.com terminates staging.xxxx.com
Rules:
  Host                                            Path  Backends
  ----                                            ----  --------
  staging.xxxx.com  
                                                  /   blue-green-staging:http (<none>)
Annotations:
  certmanager.k8s.io/cluster-issuer:  letsencrypt-staging
  kubernetes.io/ingress.class:        nginx
  kubernetes.io/tls-acme:             true
Events:
  Type    Reason             Age                  From                      Message
  ----    ------             ----                 ----                      -------
  Normal  CREATE             4m13s                nginx-ingress-controller  Ingress default/blue-green-staging
  Normal  CreateCertificate  4m13s                cert-manager              Successfully created Certificate "tls-cert-staging.xxxx.com"
  Normal  UPDATE             40s (x2 over 3m49s)  nginx-ingress-controller  Ingress default/blue-green-staging

And the logs of cert-manager are showing every two seconds:

I1023 14:32:52.348545       1 controller.go:171] certificates controller: syncing item 'default/tls-cert-staging.xxxx.com'
I1023 14:32:52.349163       1 helpers.go:194] Setting lastTransitionTime for Certificate "tls-cert-staging.xxxx.com" condition "Ready" to 2018-10-23 14:32:52.348976907 +0000 UTC m=+17582.640458718
I1023 14:32:52.354108       1 controller.go:168] ingress-shim controller: syncing item 'default/blue-green-staging'
I1023 14:32:52.354400       1 sync.go:140] Certificate "tls-cert-staging.xxxx.com" for ingress "blue-green-staging" already exists
I1023 14:32:52.354745       1 sync.go:143] Certificate "tls-cert-staging.xxxx.com" for ingress "blue-green-staging" is up to date
I1023 14:32:52.354960       1 controller.go:182] ingress-shim controller: Finished processing work item "default/blue-green-staging"
I1023 14:32:52.355444       1 controller.go:185] certificates controller: Finished processing work item "default/tls-cert-staging.xxxx.com"

Anything else we need to know?:

Environment details::

• Kubernetes version (e.g. v1.10.2): v1.10.7-gke.6
• Cloud-provider/provisioner (e.g. GKE, kops AWS, etc): GKE
• cert-manager version (e.g. v0.4.0): 0.5.0
• Install method (e.g. helm or static manifests): helm

/kind bug

[victornoel]

I tried changing the name of the secret to "tls-secret-1" and the first time the word appears in the log, it says:

I1024 07:31:11.426673       1 sync.go:140] Certificate "tls-secret-1" for ingress "blue-green-staging" already exists
I1024 07:31:11.426806       1 sync.go:143] Certificate "tls-secret-1" for ingress "blue-green-staging" is up to date

This is really strange, no?

[victornoel]

After a bit more research, I found the following, calling kubectl describe certificates.certmanager.k8s.io tls-cert-staging.xxx.com:

Name:         tls-cert-staging.xxx.com
Namespace:    default
Labels:       <none>
Annotations:  <none>
API Version:  certmanager.k8s.io/v1alpha1
Kind:         Certificate
Metadata:
  Cluster Name:        
  Creation Timestamp:  2018-10-24T07:55:06Z
  Generation:          1
  Owner References:
    API Version:           extensions/v1beta1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  Ingress
    Name:                  blue-green-staging
    UID:                   1f13c82f-d762-11e8-99af-42010a84017e
  Resource Version:        261228
  Self Link:               /apis/certmanager.k8s.io/v1alpha1/namespaces/default/certificates/tls-cert-staging.xxx.com
  UID:                     1f16d922-d762-11e8-99af-42010a84017e
Spec:
  Dns Names:
    staging.xxx.com
  Issuer Ref:
    Kind:       ClusterIssuer
    Name:       letsencrypt-staging
  Secret Name:  tls-cert-staging.xxx.com
Status:
  Conditions:
    Last Transition Time:  2018-10-24T07:56:30Z
    Message:               Resource validation failed: spec: Invalid value: "no issuer specified for Issuer '/letsencrypt-staging'": no issuer specified for Issuer '/letsencrypt-staging'
    Reason:                ConfigError
    Status:                False
    Type:                  Ready
Events:                    <none>

I suppose this is the error I was looking for, but it's not clear what is the problem.
The issuer, in the kube-system namespace (where cert-manager and nginx-ingress are installed), is defined as:

apiVersion: certmanager.k8s.io/v1alpha1
kind: ClusterIssuer
metadata:
  name: letsencrypt-staging
spec:
  # The ACME server URL
  server: https://acme-staging-v02.api.letsencrypt.org/directory
  # Email address used for ACME registration
  email: [email protected]
  # Name of a secret used to store the ACME account private key
  privateKeySecretRef:
    name: letsencrypt-staging
  # Enable the HTTP-01 challenge provider
  http01: {}

[munnerz]

Ah, it looks like your ClusterIssuer resource is invalid.

Notably, it should look more like:
.

apiVersion: certmanager.k8s.io/v1alpha1kind: ClusterIssuermetadata:
name: letsencrypt-stagingspec:

acme:
# The ACME server URL
server: https://acme-staging-v02.api.letsencrypt.org/directory
...

Your example is missing the 'acme' key, hence the error 😬

On Wed, 24 Oct 2018 at 08:58, Victor Noël [e-mail] wrote:

After a bit more research, I found the following, calling kubectl
describe certificates.certmanager.k8s.io tls-cert-staging.xxx.com:

Name: tls-cert-staging.xxx.com
Namespace: default
Labels:
Annotations:
API Version: certmanager.k8s.io/v1alpha1
Kind: Certificate
Metadata:
Cluster Name:
Creation Timestamp: 2018-10-24T07:55:06Z
Generation: 1
Owner References:
API Version: extensions/v1beta1
Block Owner Deletion: true
Controller: true
Kind: Ingress
Name: blue-green-staging
UID: 1f13c82f-d762-11e8-99af-42010a84017e
Resource Version: 261228
Self Link: /apis/certmanager.k8s.io/v1alpha1/namespaces/default/certificates/tls-cert-staging.xxx.com
UID: 1f16d922-d762-11e8-99af-42010a84017e
Spec:
Dns Names:
staging.xxx.com
Issuer Ref:
Kind: ClusterIssuer
Name: letsencrypt-staging
Secret Name: tls-cert-staging.xxx.com
Status:
Conditions:
Last Transition Time: 2018-10-24T07:56:30Z
Message: Resource validation failed: spec: Invalid value: "no issuer specified for Issuer '/letsencrypt-staging'": no issuer specified for Issuer '/letsencrypt-staging'
Reason: ConfigError
Status: False
Type: Ready
Events:

I suppose this is the error I was looking for, but it's not clear what is
the problem.
The issuer, in the kube-system namespace (where cert-manager and
nginx-ingress are installed), is defined as:

apiVersion: certmanager.k8s.io/v1alpha1kind: ClusterIssuermetadata:
name: letsencrypt-stagingspec:
# The ACME server URL
server: https://acme-staging-v02.api.letsencrypt.org/directory
# Email address used for ACME registration
email: [email protected]
# Name of a secret used to store the ACME account private key
privateKeySecretRef:
name: letsencrypt-staging
# Enable the HTTP-01 challenge provider
http01: {}

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/jetstack/cert-manager/issues/992#issuecomment-432552811,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAMbP7zZeXslCDbQDGfl2hD674Zg-qaSks5uoB22gaJpZM4X1u-q
.

[victornoel]

@munnerz arg, I'm ashamed :)

Thanks!

[mihairadulescu]

@victornoel im more assamed you are asshamed cas im the same typo as you :)