Aws-load-balancer-controller: Several `aws-alb-ingress-controller` in one namespace

Created on 7 Aug 2019  路  6Comments  路  Source: kubernetes-sigs/aws-load-balancer-controller

When I deploy several controllers in to one namespace, first controller is up and running but second is stuck with that issue:

AWS ALB Ingress controller
  Release:    v1.1.2
  Build:      git-cc1c5971
  Repository: https://github.com/kubernetes-sigs/aws-alb-ingress-controller.git
-------------------------------------------------------------------------------

W0807 15:13:05.886110       1 client_config.go:549] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0807 15:13:05.947972       1 :0] kubebuilder/controller "level"=0 "msg"="Starting EventSource"  "controller"="alb-ingress-controller" "source"={"Type":{"metadata":{"creationTimestamp":null}}}
I0807 15:13:05.948357       1 :0] kubebuilder/controller "level"=0 "msg"="Starting EventSource"  "controller"="alb-ingress-controller" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{},"status":{"loadBalancer":{}}}}
I0807 15:13:05.948441       1 :0] kubebuilder/controller "level"=0 "msg"="Starting EventSource"  "controller"="alb-ingress-controller" "source"=
I0807 15:13:05.948771       1 :0] kubebuilder/controller "level"=0 "msg"="Starting EventSource"  "controller"="alb-ingress-controller" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{},"status":{"loadBalancer":{}}}}
I0807 15:13:05.948820       1 :0] kubebuilder/controller "level"=0 "msg"="Starting EventSource"  "controller"="alb-ingress-controller" "source"=
I0807 15:13:05.949035       1 :0] kubebuilder/controller "level"=0 "msg"="Starting EventSource"  "controller"="alb-ingress-controller" "source"={"Type":{"metadata":{"creationTimestamp":null}}}
I0807 15:13:05.949558       1 :0] kubebuilder/controller "level"=0 "msg"="Starting EventSource"  "controller"="alb-ingress-controller" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{},"status":{"daemonEndpoints":{"kubeletEndpoint":{"Port":0}},"nodeInfo":{"machineID":"","systemUUID":"","bootID":"","kernelVersion":"","osImage":"","containerRuntimeVersion":"","kubeletVersion":"","kubeProxyVersion":"","operatingSystem":"","architecture":""}}}}
I0807 15:13:05.949982       1 leaderelection.go:205] attempting to acquire leader lease  ingress-controllers/ingress-controller-leader-alb...

My configurations for controllers are:
first:

  - name: "alb-ingress-1"
    namespace: "ingress-controllers"
    labels:
      chart: "aws-alb-ingress-controller"
      repo: "incubator"
      component: "ingress"
      namespace: "ingress-controllers"
      vendor: "kubernetes"
      default: "false"
    chart: "incubator/aws-alb-ingress-controller"
    version: "0.1.4"
    wait: true
    installed: "true"
    values:
    - rbac:
        ## If true, create & use RBAC resources
        ##
        create: '{{ env "RBAC_ENABLED" | default "true" }}'
        serviceAccountName: "alb-ingress-controller-1"

      clusterName: '{{ env "KOPS_CLUSTER_NAME" }}'

      scope:
        ingressClass: "alb-1"

Second:

  - name: "alb-ingress-2"
    namespace: "ingress-controllers"
    labels:
      chart: "aws-alb-ingress-controller"
      repo: "incubator"
      component: "ingress"
      namespace: "ingress-controllers"
      vendor: "kubernetes"
      default: "false"
    chart: "incubator/aws-alb-ingress-controller"
    version: "0.1.4"
    wait: true
    installed: "true"
    values:
    - rbac:
        ## If true, create & use RBAC resources
        ##
        create: '{{ env "RBAC_ENABLED" | default "true" }}'
        serviceAccountName: "alb-ingress-controller-1"

      clusterName: '{{ env "KOPS_CLUSTER_NAME" }}'

      scope:
        ingressClass: "alb-2"

If I set the different namespaces for those controllers - they start working well

Most helpful comment

Actually...I just figured it out. You can use the election-id option to set the name of the config map that is used for leader election.

https://github.com/kubernetes-sigs/aws-alb-ingress-controller/blob/master/cmd/options.go#L82

This option isnt/wasnt documented anywhere which is why it took some digging to find it

All 6 comments

Have you gotten any resolution on this?

I would think that's the expected behavior since there would only be 1 leader at a time?

Yes, but I set different controllers (different ingressClass), so I think that it happens because controller creates config map with the same name in the namespace even if that is different controller. That config map controller uses for leader leasing.
I think it should be one config map per ingressClass, it will provide the ability to have more then one controller in one namespace.
!!! E.g:
namespace: test-1
IngressClass: ig1
IngressClass: ig2
Ig1 and Ig2 - ingress controllers with different IngressClass.
It should work, but currently - does not, because just one config map is created
and second ig will waiting leader election

+1 for this. How can we control the name of the config map used for leader election?

The NGINX Ingress Controller allows you to do this with the -leader-election-lock-name option: https://docs.nginx.com/nginx-ingress-controller/configuration/global-configuration/command-line-arguments/

Not having this ability causes issues if you want to run two ingress controllers in the same namespace

Actually...I just figured it out. You can use the election-id option to set the name of the config map that is used for leader election.

https://github.com/kubernetes-sigs/aws-alb-ingress-controller/blob/master/cmd/options.go#L82

This option isnt/wasnt documented anywhere which is why it took some digging to find it

@csansone-handy Looks like this is an answer on my question!
Thank you so much!
Will let you know when I test this solution

I've tested it and I confirm that it works as expected!
Thanks @csansone-handy

Was this page helpful?
0 / 5 - 0 ratings

Related issues

jcderr picture jcderr  路  3Comments

ishaannarang picture ishaannarang  路  5Comments

rootd00d picture rootd00d  路  4Comments

joseppla picture joseppla  路  5Comments

sawanoboly picture sawanoboly  路  5Comments