Amazon-vpc-cni-k8s: Pod Network Policies with vpc-cni

[tirumerla]

What happened:

I have applied pod network policies for all of my namespaces and individual pods. None of them have impact on restricting the traffic, even though i blocked egress to all traffic except to specific IP i still can access the internet ( eg. amazon.com, google.com etc.. ) from the individual pods. I was wondering if vpc-cni ( aws-node daemon set ) is enough to enforce network policies or do i need calico to enforce these network policies.

Here's the example policy i have

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: jupyternotebook
spec:
  podSelector: {}
  policyTypes:
  - Egress
  egress: []
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: jupyternotebook-access
  namespace: jupyternotebook
spec:
  podSelector: 
    matchLabels:
      app: jupyternotebook
  policyTypes:
  - Ingress
  - Egress
  ingress:
    - ports:
        - port: 443
          protocol: TCP
      from:
        - namespaceSelector:
            matchLabels:
              namespace: traefik
          podSelector:
            matchLabels:
              app.kubernetes.io/instance: traefik
              app.kubernetes.io/managed-by: Helm
              app.kubernetes.io/name: traefik
              helm.sh/chart: traefik-9.1.1
  egress:
    - to:
        - ipBlock:
            cidr: <dest_ip>/32
      ports:
        - port: 4000
          protocol: TCP
        - port: 53
          protocol: TCP
        - port: 53
          protocol: UDP

Appreciate your help. Thanks.

Environment:

• Kubernetes version (use kubectl version): EKS - v1.17.9
• CNI Version: v1.6.3
• OS (e.g: cat /etc/os-release): Amazon Linux 2
• Kernel (e.g. uname -a): Linux 4.14.193-149.317.amzn2.x86_64 #1 SMP x86_64 GNU/Linux

[abhipth]

Hi @tirumerla EKS CNI Plugin does not enforce network policies. You may consider installing Calico on your cluster. Alternatively, we have recently launched Security Group Per Pod that allocates an Elastic Network Interface (ENI) Per Pod allowing you to control traffic using the native Security Groups provided by EC2. The latter solution would not require installing Calico or applying network policies, instead you can use Security Group Policy as mentioned in the linked document.

[tirumerla]

Hi @abhipth Thanks for your response. Looks like t3 instance family is not one of the supported instances for security group per pod which i'm using currently. Looks like the only option for me is to install calico.

[abhipth]

Yes that's correct only Nitro based instance families can currently support this feature. Calico should solve your use case too. Please feel free to re open the issue if you have further question or encounter a new issue.

[tirumerla]

Thanks @abhipth. I have got calico installed and deploy all my network policies. I'm running into network policy precedence issue. For the network policy i posted above ^^^ -- even though i have got allow policy right after deny policy i see that deny policy is taking precedence over allow policy ( egress traffic is blocked from individual pods that have _allow egress policy set_ ). Based on the answer here, I assume allow policy should still work. Wondering if i'm missing something.

[tirumerla]

Update - Solved the issue. There was label mismatch between pods and svc which was causing the issue.