Lokomotive: Add calico-kube-controllers to calico deployment

[invidian]

To be able to use K8s NetworkPolicy object with Calico and later on benefit from automatic host endpoint.

References:

• https://github.com/projectcalico/kube-controllers/pull/458
• https://github.com/projectcalico/kube-controllers
• https://docs.projectcalico.org/reference/kube-controllers/configuration

[rata]

To add to this, calico-hostendpoint-controller component is probably possible to be removed after this (the AutomaticHostEndpoint uses * for the interface, so it might affect the VPN pod, for example. We need to be careful)

[surajssd]

Adding tests for testing VPN before enabling this sounds like a way to go!

[alban]

I can give this a try.

I am trying to understand the history: I see in assets/components/calico-hostendpoint-controller/manifests/deployment.yaml that we use quay.io/kinvolk/calico-hostendpoint-controller:v0.0.2. It is the version number referred in https://github.com/kinvolk/calico-hostendpoint-controller/commit/878b56cd423e235598adb2c8432ac4ddf3704a9b#diff-45ed3b7096bcaff1633c99d50b30e99aR27 , before it was merged into https://github.com/projectcalico/kube-controllers.

[invidian]

@alban this issue is about adding upstream calico-kube-controller deployment to our Calico setup. calico-hostendpoint-controller stays for now I think (though some functionality, like removing old hostendpoint objects could be removed), until https://github.com/projectcalico/kube-controllers/pull/458 is released.

[alban]

To be able to use K8s NetworkPolicy object with Calico

I tested that network policies seems to work fine without changes.

In two terminals, I start two pods:

kubectl run --restart=Never --image=ubuntu shell1 -ti --rm -- /bin/bash
kubectl run --restart=Never --image=ubuntu shell2 -ti --rm -- /bin/bash

After installing tools with apt-get update && apt-get install net-tools netcat, I test network connectivity between the two pods:

nc -l -p 8080       # In terminal 1
nc  10.2.162.3 8080 # In terminal 2

I tried with and without the network policy from network-policy-default-deny-ingress.yaml:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-ingress
  namespace: default
spec:
  podSelector: {}
  policyTypes:
  - Ingress

Results:

• Without the network policy, I have pod-to-pod connectivity
• With the network policy, I don't have pod-to-pod connectivity

Edit 10.03.2020: I also tested this NetworkPolicy and it works:

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  namespace: kube-system
  name: coredns-allow-dns-from-all-namespaces
spec:
  podSelector:
    matchLabels:
      k8s-app: coredns
  ingress:
  - ports:
    - port: 53
      protocol: UDP
    - port: 53
      protocol: TCP
    from:
    - namespaceSelector: {}

[alban]

The Calico datastore can be configured to use either the Kubernetes backend or the etcd backend. In Lokomotive, the Calico datastore is configured to use the Kubernetes backend:

$ kubectl get pod -n kube-system -l'k8s-app=calico-node' -o yaml|grep -A 1 DATASTORE_TYPE
      - name: DATASTORE_TYPE
        value: kubernetes

According to the controllers documentation:

This controller [policy controller, workload endpoint controller, namespace controller and service account controller] is only valid when using etcd as the Calico datastore.

So we don't need to add calico-kube-controllers in our deployments.

[invidian]

What about automatic HostEndpoint objects coming in the next release of the Calico? I think we still need the controller then?

[invidian]

@alban I assigned myself, I guess you're not working on it.