Aws-load-balancer-controller: Issues with certificates

[mlankenau]

We are getting issues when the ingres is updating/adding new subdomains. We have one ALB that is shared for all our apps/microservices. But it uses two SSL certificates. I am not sure if that error is because of the two certificates (comma separated) or because of permissions.

kubebuilder/controller "msg"="Reconciler error" "error"="failed to reconcile listeners due to failed to reconcile listener due to CertificateNotFound: Certificate 'arn:aws:acm:eu-central-1:XXXXXXXXXXX:certificate/14a11773-91d6-483e-9d69-766bd2d1feb9,arn:aws:acm:eu-central-1:XXXXXXXXXXX:certificate/e6b12d18-ae5d-4992-9bb1-b5e7564ba7e5' not found\n\tstatus code: 400, request id: 793e4748-5b70-11e9-bbee-631615d90c01" "controller"="alb-ingress-controller" "request"={"Namespace":"default","Name":"test-price-api-ingres"}

the image of the controller is: docker.io/amazon/aws-alb-ingress-controller:v1.1.0

[M00nF1sh]

Hi，currently the auto-discover feature only looks for the domainName(common name field) in certificate instead of SAN.
You can manually specify the certificate ARNs with alb.ingress.kubernetes.io/certificate-arn: arn1,arn2,arn3 annotation.
I'll create an PR to improve the auto-discover cert feature.

[mlankenau]

Hi, I already list the arns in the Ingres like this:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: test-price-api-ingres
annotations:
kubernetes.io/ingress.class: alb
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/target-type: instance
alb.ingress.kubernetes.io/subnets: subnet-XXXXXXXXXXXXXX,subnet-XXXXXXXXX,subnet-XXXXXXXXXXXX
alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:eu-central-1:XXXXXXXXX:certificate/14a11773-91d6-483e-9d69-766bd2d1feb9,arn:aws:acm:eu-central-1:XXXXXXXX:certificate/e6b12d18-ae5d-4992-9bb1-b5e7564ba7e5
alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS": 443}]'
spec:
...

Is it maybe related to the older Version that we use? Is it safe to just update to v1.1.2 by changing the deployment of the ingres?

[M00nF1sh]

the multiple certificate feature is available in v1.1.0, i guess you are below that.
It's safe to upgrade to v1.1.2 by changing the deployment yaml only when you are above v1.0.0. What's your current version?

[mlankenau]

We run v1.1.0

[M00nF1sh]

Sorry, typo in my last comment: the multiple certificate feature is available in v1.1.1 instead of v.1.1.0

You can upgrade by following step:

1. ensure you have all iam permissions for worker node. (a few new permissions are added in v1.1.1 and v1.1.2)
2. change the ALB ingress controller version from v1.1.0 to v1.1.2

Full changelog can be viewed in https://github.com/kubernetes-sigs/aws-alb-ingress-controller/releases

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[k]

I get

[aws-alb-ingress-controller-55f95bc8b8-wl7gd] E0716 03:28:13.756824       1 :0] kubebuilder/controller "msg"="Reconciler error" "error"="failed to reconcile listeners due to failed to create listener due to CertificateNotFound: Certificate 'arn:aws:acm:us-west-2:069289935426:certificate/0f6f2a07-fb3d-4214-83d8-9f12853111dd' not found\n\tstatus code: 400, request id: be4895c4-a779-11e9-a0e4-b3c5309a14dd"  "controller"="alb-ingress-controller" "request"={"Namespace":"geo","Name":"geo"}

even though I ctrl-F'd for that arn in the same region as my eks cluster and it matches. Any idea why it can't find it? I gave the role full ACM permissions as well

[M00nF1sh]

@k Is the certificate validated? invalid certificated cannot be configured on alb.
You can test it by manually create an HTTPS listener and check the certificates available

[errordeveloper]

@M00nF1sh so to be clear, you are saying that this error often means user has an invalid (e.g. self-signed) certificate?

[M00nF1sh]

@errordeveloper imported self-signed certificate is fine. It should be an certificate created an ACM, but is not validated via email/DNS.

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[errordeveloper]

/reopen

I don't think this was solved, the error message should be more helpful.

[k8s-ci-robot]

@errordeveloper: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to this:

/reopen

I don't think this was solved, the error message should be more helpful.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.