Aws-load-balancer-controller: Duplicated rules

[yawboateng]

I have deleted and recreated my ingress object multiple times and I just noticed that in AWS under the alb rules, there are a lot of duplication.

including going into aws and manually deleting all the albs and target groups, but each time the controller recreates the alb, the rules are duplicated

[joshrosso]

Hi @yawboateng

Could you send me a screenshot of what the rule duplication looks like from the AWS UI?

Also, could you provide a sample ingress resource manifest?

[yawboateng]

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: cloud-statistics-bot-alb
  namespace: auth
  kubernetes.io/ingress.class: "alb"
spec:
  rules:
  - host: apps.test.com
    http:
      paths:
      - path: /
        backend:
          serviceName: app-bot
          servicePort: 80
      - path: {{ .Values.apiPath }}/internetAddresses/mailServer/*
        backend:
          serviceName: nginx-ingress-controller
          servicePort: 443
      - path: {{ .Values.apiPath }}/user/*
        backend:
          serviceName: nginx-ingress-controller
          servicePort: 443
      - path: {{ .Values.apiPath }}/companies
        backend:
          serviceName: nginx-ingress-controller
          servicePort: 443
      - path: {{ .Values.apiPath }}/users/mailServer/*
        backend:
          serviceName: nginx-ingress-controller
          servicePort: 443
      - path: {{ .Values.apiPath }}/mailServers/intersection
        backend:
          serviceName: nginx-ingress-controller
          servicePort: 443
      - path: {{ .Values.apiPath }}/users/travelerServer/*
        backend:
          serviceName: nginx-ingress-controller
          servicePort: 443
      - path: {{ .Values.apiPath }}/internetAddresses/travelerServer/*
        backend:
          serviceName: nginx-ingress-controller
          servicePort: 443
      - path: {{ .Values.apiPath }}/travelerServers/union
        backend:
          serviceName: nginx-ingress-controller
          servicePort: 443
      - path: /api/v1/oauth2*
        backend:
          serviceName: "oauth2-client"
          servicePort: 80

[joshrosso]

Thanks for the details @yawboateng, shooting to look into this over the weekend.

[yawboateng]

So i noticed the pattern being, everytime i update my ingress and add a new rule, it appends the new rule plus all the other existing rules to the end of the current rules

[joshrosso]

Thanks for the update on this pattern @yawboateng, sounds pretty easy to solve and knock out. Will update soon.

[joshrosso]

@yawboateng: I'm struggling to reproduce this on my end.

I've deployed a manifest and then added and removed rules directly using kubectl edit, but cannot seem to get it to duplicate any rules.

As a sanity check, what version of the controller are you using?

In my tests, I was using quay.io/coreos/alb-ingress-controller:b1110dca

Full version list: https://quay.io/repository/coreos/alb-ingress-controller?tag=latest&tab=tags

[yawboateng]

Hi @joshrosso,
I was using 1adffb1d at that time. currently running latest and haven't see any duplicates yet

[cmaloney]

I've run into this a couple times. I think it happens when more than two rules are removed at once.

https://github.com/coreos/alb-ingress-controller/blob/master/controller/alb/rules.go#L13

loops through the rules, and deletes them as they're found. If it's already deleted one rule, and goes to delete another, it doesn't account for the fact that the underlying rule array has already shifted by one, so it's delete location isn't quite right. The TargetGroups code has a similar bug which resulted in a crash/panic for me after deleting multiple target groups simultaneously (There was an out of array index)

[joshrosso]

Yup, seems right, We will do a lookup on the rule's index instead.

[bigkraig]

@yawboateng just merged the fix committed back in may, LMK if you still see the issue

[yawboateng]

no issues so far, thanks...closing