Falco: Privileged container do not get recogniced as such (

[PhilipSchmid]

Describe the bug

When I start a privileged container the "Launch Privileged Container" default rule does not get triggered. Because of this, I tried to use a modified version of this rule in order to see, if Falco is even able to detect that this example container is started as a privileged one:

- macro: container_started
  condition: ((evt.type = container) or (spawned_process and proc.vpid=1))

- rule: Launch Privileged Container
  desc: Detect the initial process started in a privileged container. Exceptions are made for known trusted images.
  condition: >
    container_started and container
  output: Privileged container started (priv=%container.privileged user=%user.name user_loginuid=%user.loginuid command=%proc.cmdline %container.info image=%container.image.repository:%container.image.tag)
  priority: INFO
  tags: [container, cis, mitre_privilege_escalation, mitre_lateral_movement]

It's not (see "container.privileged":false):

{"priority":"Notice","rule":"Launch Privileged Container","time":"2020-11-04T11:50:49.740339716Z", "output_fields": {"container.id":"f4318ef0227c","container.image.repository":null,"container.image.tag":null,"container.privileged":false,"evt.time":1604490649740339716,"k8s.ns.name":null,"k8s.pod.name":null,"proc.cmdline":"pause","user.loginuid":-1,"user.name":"root"}}
{"priority":"Notice","rule":"Launch Privileged Container","time":"2020-11-04T11:50:51.108031343Z", "output_fields": {"container.id":"14108da41f70","container.image.repository":null,"container.image.tag":null,"container.privileged":false,"evt.time":1604490651108031343,"k8s.ns.name":null,"k8s.pod.name":null,"proc.cmdline":"sleep 3600","user.loginuid":-1,"user.name":"root"}}

How to reproduce it

echo '
apiVersion: v1
kind: Pod
metadata:
  name: falcotest
  namespace: default
spec:
  serviceAccountName: falcotest
  containers:
  - args:
    - "/bin/sleep"
    - "3600"
    image: busybox:1.32.0
    name: falcotest
    securityContext:
      privileged: true
' | kubectl apply -f -

(ServiceAccount falcotest is referring to a PSP with privileged set to true.)

Container gets created:

$ kubectl get pods -n default
NAME        READY   STATUS    RESTARTS   AGE
falcotest   1/1     Running   0          6m32s

Expected behaviour

The "Launch Privileged Container" default rule should trigger since it matches for container.privileged=true: https://github.com/falcosecurity/falco/blob/294804daf436b4d146731322f46a99e2736feee0/rules/falco_rules.yaml#L1959-L1968

Environment

• Falco version: 0.26.1
  
• System info:

{
  "machine": "x86_64",
  "nodename": "node1",
  "release": "4.15.0-117-generic",
  "sysname": "Linux",
  "version": "#118~16.04.1-Ubuntu SMP Sat Sep 5 23:35:06 UTC 2020"
}

• Cloud provider or hardware configuration: PKS 1.8.2
• OS: Kubernetes
  
• Kernel: 4.15.0-117
  
• Installation method: Kubernetes

Additional context

Helm chart version: 1.5.0

Thanks & regards,
Philip

[leogr]

{"priority":"Notice","rule":"Launch Privileged Container","time":"2020-11-04T11:50:49.740339716Z", "output_fields": {"container.id":"f4318ef0227c","container.image.repository":null,"container.image.tag":null,"container.privileged":false,"evt.time":1604490649740339716,"k8s.ns.name":null,"k8s.pod.name":null,"proc.cmdline":"pause","user.loginuid":-1,"user.name":"root"}}
{"priority":"Notice","rule":"Launch Privileged Container","time":"2020-11-04T11:50:51.108031343Z", "output_fields": {"container.id":"14108da41f70","container.image.repository":null,"container.image.tag":null,"container.privileged":false,"evt.time":1604490651108031343,"k8s.ns.name":null,"k8s.pod.name":null,"proc.cmdline":"sleep 3600","user.loginuid":-1,"user.name":"root"}}

I see container/k8s metadata is missing too. It seems to me that Falco was not able to fetch information from the k8s api-server.

Could you provide additional details about your Kubernetes installation, please?

Which is the container runtime?

Also, check if Falco can access what is needed to fetch that info.

[PhilipSchmid]

Hi @leogr

Thanks for your answer.

The Falco DaemonSet is started with the following args:

      - args:
        - /usr/bin/falco
        - --cri
        - /run/containerd/containerd.sock
        - -K
        - /var/run/secrets/kubernetes.io/serviceaccount/token
        - -k
        - https://$(KUBERNETES_SERVICE_HOST)
        - -pk

Also the kube-apiserver seems to be reachable from within a falco Pod using the falco ServiceAccount token from the secret:

[user@host:~]$ kubectl get secret $(kubectl get serviceaccount falco -o jsonpath='{.secrets[0].name}') -o jsonpath='{.data.token}' | base64 --decode        
ey....IQ

[user@host:~]$ kubectl exec -it falco-cxm6b -- bash

# The token inside the container is the same:
root@2b67af75-69e0-4d99-9458-c1528a2c9af7:/# cat /var/run/secrets/kubernetes.io/serviceaccount/token 
ey....IQ
root@2b67af75-69e0-4d99-9458-c1528a2c9af7

root@2b67af75-69e0-4d99-9458-c1528a2c9af7:/# curl -k https://$KUBERNETES_SERVICE_HOST/api --header "Authorization: Bearer ey....IQ"
{
  "kind": "APIVersions",
  "versions": [
    "v1"
  ],
  "serverAddressByClientCIDRs": [
    {
      "clientCIDR": "0.0.0.0/0",
      "serverAddress": "10.2.3.4:8443"
    }
  ]
}root@2b67af75-69e0-4d99-9458-c1528a2c9af7:/#

Here some more information about the runtime:

[user@host:~]$ docker info
Client:
 Debug Mode: false

Server:
...
 Server Version: 19.03.5
...
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: b34a5c8af56e510852c35414db4c1f4fa6172339
 runc version: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
 init version: fec3683
...
 Kernel Version: 4.15.0-117-generic
 Operating System: Ubuntu 16.04.7 LTS
 OSType: linux
 Architecture: x86_64
...
 Docker Root Dir: /var/vcap/store/docker/docker
...
 Product License: Community Engine

What other information would be useful for you to debug?

The Falco K8s Audit feature is not enabled (Helm chart option auditLog.enabled: false) but I do not think this rule would need this since its not from k8s_audit_rules.yaml, right?

Thanks!

Regards,
Philip

[leogr]

Thank you, @PhilipSchmid

I suspect the root cause is the missing metadata. Btw, it might be related to something like this other issue: https://github.com/falcosecurity/falco/issues/1421.
If you noticed the same behavior (missing container/k8s info) with other rules, I would say that's the root cause.

Other than that, you confirmed that essential things are ok, so I don't have other questions now. I will try to reproduce the problem.

About the k8s Audit feature, you're right. It's not needed in this case.

Thank you again!

[PhilipSchmid]

Hi @leogr,

Thanks for your hint with the DS container arguments!! That exactly where the problem was.. PKS uses custom containerd and docker socket paths. The following Falco Helm chart values resolve the issue:

containerd:
  socket: /var/run/docker/containerd/containerd.sock
docker:
  socket: /var/vcap/sys/run/docker/docker.sock

Regards,
Philip