Kube-router: BGP Advertisement of External-IPs on LoadBalancers

[carpenike]

What happened?
I'm leveraging Kube-router along with Cilium to advertise internal cluster IPs via BGP to an edge device. Cilium provides CNI capabilities and I've validated that it's able to expose ClusterIPs, Pod IPs, and External-IPs assigned to either LoadBalancers or ClusterIPs. However, Kube-Router does not seem to pick up or advertise the External-IP address if it is assigned to a service of type = LoadBalancer. Only works when service type = ClusterIP.

What did you expect to happen?
I would anticipate that Kube-Router should discover External-IPs on both svc type: LoadBalancer as well as svc type: ClusterIP.

How can we reproduce the behavior you experienced?
Steps to reproduce the behavior:

1. Deploy Kube-Router with the following config:

      - name: kube-router
        image: docker.io/cloudnativelabs/kube-router:v1.1.0-rc1
        imagePullPolicy: Always
        args:
        - "--run-router=true"
        - "--run-firewall=false"
        - "--run-service-proxy=false"
        - "--bgp-graceful-restart=true"
        - "--enable-cni=false"
        - "--enable-pod-egress=false"
        - "--enable-ibgp=true"
        - "--enable-overlay=false"
        - "--peer-router-ips=10.20.0.1"
        - "--peer-router-asns=64512"
        - "--cluster-asn=64513"
        - "--advertise-cluster-ip=true"
        - "--advertise-external-ip=true"
        - "--advertise-loadbalancer-ip=true"

1. Deploy a simple service where LoadBalancer is defined as type and an ExternalIP is defined:

apiVersion: v1
kind: Service
metadata:
  name: httpbin
  labels:
    app: httpbin
spec:
  type: LoadBalancer
  externalIPs:
    - 10.45.10.75
  ports:
    - name: http
      port: 8000
      targetPort: 80
  selector:
    app: httpbin

1. External-IP will not be advertised to BGP peers.

*Screenshots / Architecture Diagrams / Network Topologies *
If applicable, add those here to help explain your problem.

* System Information (please complete the following information):*

• Kube-Router Version (kube-router --version): v1.1.0-rc1
• Kube-Router Parameters: See Above
• Kubernetes Version (kubectl version) : 1.19.2
• Cloud Type: On premise
• Kubernetes Deployment Type: kubeadm
• Kube-Router Deployment Type: Daemonset
• Cluster Size: 4 Nodes

[murali-reddy]

thanks for reporting @carpenike the issue.

https://github.com/cloudnativelabs/kube-router/blob/v1.1.0-rc1/pkg/controllers/routing/ecmp_vip.go#L318 should include services of LoadBalancer as well. Will get it fixed.

[carpenike]

@murali-reddy -- Awesome! Looks straightforward, do you want a PR for that 1 line fix? :)

[murali-reddy]

that would be nice @carpenike Please go ahead.

[carpenike]

Awesome. @onedr0p and I just got it working in our own compiled kube-router. I imagine he'll submit the PR soon.

Appreciate you identifying the problem so quickly.