Fluentd-kubernetes-daemonset: RBAC policy example required
Hello,
I tried deploying fluent/fluentd-kubernetes-daemonset:v0.12-alpine-s3 into an RBAC enabled Kubernetes cluster and received this error:
```
2018-03-19 17:51:33 +0000 [error]: config error file="/fluentd/etc/fluent.conf" error="Exception encountered fetching metadata from Kubernetes API endpoint: pods is forbidden: User \"system:serviceaccount:kube-system:default\" cannot list pods at the cluster scope ({\"kind\":\"Status\",\"apiVersion\":\"v1\",\"metadata\":{},\"status\":\"Failure\",\"message\":\"pods is forbidden: User \\"system:serviceaccount:kube-system:default\\" cannot list pods at the cluster scope\",\"reason\":\"Forbidden\",\"details\":{\"kind\":\"pods\"},\"code\":403}\n)"
````
jpds
All 9 comments
I have the same issue. Anyone has a solution ?
medanasslim
on 22 Mar 2018
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
k8s-app: fluentd
name: fluentd
namespace: kube-system
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: fluentd-clusterrole
rules:
- apiGroups:
- ""
resources:
- "namespaces"
- "pods"
verbs:
- "list"
- "get"
- "watch"
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: fluentd-clusterrole
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: fluentd-clusterrole
subjects:
- kind: ServiceAccount
name: fluentd
namespace: kube-system
rbtr
on 22 Mar 2018
Hi, I have the following error when creating the clusterrole. Any idea what I'm missing ?
$ kubectl create -f clusterrole.yaml
Error from server (Forbidden): error when creating "test-clusterrole.yaml":
clusterroles.rbac.authorization.k8s.io "fluentd-clusterrole" is forbidden: attempt to grant extra privileges:
[PolicyRule{Resources:["namespaces"], APIGroups:[""], Verbs:["list"]}
PolicyRule{Resources:["namespaces"], APIGroups:[""], Verbs:["get"]}
PolicyRule{Resources:["namespaces"], APIGroups:[""], Verbs:["watch"]}
PolicyRule{Resources:["pods"], APIGroups:[""], Verbs:["list"]}
PolicyRule{Resources:["pods"], APIGroups:[""], Verbs:["get"]}
PolicyRule{Resources:["pods"], APIGroups:[""], Verbs:["watch"]}] user=&{[email protected]
[system:authenticated] map[user-assertion.cloud.google.com:
[yyyyy=]]} ownerrules=
[PolicyRule{Resources:["selfsubjectaccessreviews" "selfsubjectrulesreviews"], APIGroups:
["authorization.k8s.io"], Verbs:["create"]} PolicyRule{NonResourceURLs:["/api" "/api/*" "/apis" "/apis/*" "/healthz" "/swagger-2.0.0.pb-v1" "/swagger.json" "/swaggerapi" "/swaggerapi/*" "/version"], Verbs:["get"]}] ruleResolutionErrors=[]
lucj
on 23 Nov 2018
My bad...
my account was missing a cluster-admin role.
kubectl create clusterrolebinding cluster-admin-binding \
--clusterrole cluster-admin \
--user $(gcloud config get-value account)
This used to work well for me till few days when deploying this on a Digital Ocean Kubernets cluster.
helm version
Client: &version.Version{SemVer:"v2.11.0", GitCommit:"2e55dbe1fdb5fdb96b75ff144a339489417b146b", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.11.0", GitCommit:"2e55dbe1fdb5fdb96b75ff144a339489417b146b", GitTreeState:"clean"}
kubectl version
Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.1", GitCommit:"b7394102d6ef778017f2ca4046abbaa23b88c290", GitTreeState:"clean", BuildDate:"2019-04-08T17:11:31Z", GoVersion:"go1.12.1", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.5", GitCommit:"2166946f41b36dea2c4626f90a77706f426cdea2", GitTreeState:"clean", BuildDate:"2019-03-25T15:19:22Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}
logs from fluentd pod:
Unexpected error start_namespace_watch: Exception encountered setting up namespace watch from Kubernetes API v1 endpoint https://10.23.0.1:443/api: namespaces is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "namespaces" in API group "" at the cluster scope ({"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"namespaces is forbidden: User \"system:serviceaccount:kube-system:default\" cannot list resource \"namespaces\" in API group \"\" at the cluster scope","reason":"Forbidden","details":{"kind":"namespaces"},"code":403}
)
ctyjrsy
on 3 May 2019
Use kubectl create clusterrolebinding add-on-cluster-admin \
--clusterrole=cluster-admin \
--serviceaccount=kube-system:default
ctyjrsy
on 3 May 2019
Instead of giving cluster-admin to the kube-system:default serviceaccount, you should modify the daemonset to use the fluentd serviceaccount created in rbtr's example. Here's a complete example for syslog:
apiVersion: v1
kind: ServiceAccount
metadata:
name: fluentd
namespace: kube-system
labels:
k8s-app: fluentd-logging
version: v1
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: fluentd
namespace: kube-system
rules:
- apiGroups:
- ""
resources:
- pods
- namespaces
verbs:
- get
- list
- watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: fluentd
roleRef:
kind: ClusterRole
name: fluentd
apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
name: fluentd
namespace: kube-system
---
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
name: fluentd
namespace: kube-system
labels:
k8s-app: fluentd-logging
version: v1
spec:
template:
metadata:
labels:
k8s-app: fluentd-logging
version: v1
spec:
serviceAccount: fluentd
serviceAccountName: fluentd
tolerations:
- key: node-role.kubernetes.io/master
effect: NoSchedule
containers:
- name: fluentd
image: fluent/fluentd-kubernetes-daemonset:v1-debian-syslog
env:
- name: SYSLOG_HOST
value: "sysloghost"
- name: SYSLOG_PORT
value: "514"
- name: SYSLOG_PROTOCOL
value: "udp"
resources:
limits:
memory: 200Mi
requests:
cpu: 100m
memory: 200Mi
volumeMounts:
- name: varlog
mountPath: /var/log
- name: varlibdockercontainers
mountPath: /var/lib/docker/containers
readOnly: true
terminationGracePeriodSeconds: 30
volumes:
- name: varlog
hostPath:
path: /var/log
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containers
kylebevans
on 24 Sep 2019
not mentioned on https://docs.fluentd.org/container-deployment/kubernetes
masterkain
on 26 Oct 2019
Instead of giving cluster-admin to the kube-system:default serviceaccount, you should modify the daemonset to use the fluentd serviceaccount created in rbtr's example. Here's a complete example for syslog:
apiVersion: v1 kind: ServiceAccount metadata: name: fluentd namespace: kube-system labels: k8s-app: fluentd-logging version: v1 --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRole metadata: name: fluentd namespace: kube-system rules: - apiGroups: - "" resources: - pods - namespaces verbs: - get - list - watch --- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: fluentd roleRef: kind: ClusterRole name: fluentd apiGroup: rbac.authorization.k8s.io subjects: - kind: ServiceAccount name: fluentd namespace: kube-system --- apiVersion: extensions/v1beta1 kind: DaemonSet metadata: name: fluentd namespace: kube-system labels: k8s-app: fluentd-logging version: v1 spec: template: metadata: labels: k8s-app: fluentd-logging version: v1 spec: serviceAccount: fluentd serviceAccountName: fluentd tolerations: - key: node-role.kubernetes.io/master effect: NoSchedule containers: - name: fluentd image: fluent/fluentd-kubernetes-daemonset:v1-debian-syslog env: - name: SYSLOG_HOST value: "sysloghost" - name: SYSLOG_PORT value: "514" - name: SYSLOG_PROTOCOL value: "udp" resources: limits: memory: 200Mi requests: cpu: 100m memory: 200Mi volumeMounts: - name: varlog mountPath: /var/log - name: varlibdockercontainers mountPath: /var/lib/docker/containers readOnly: true terminationGracePeriodSeconds: 30 volumes: - name: varlog hostPath: path: /var/log - name: varlibdockercontainers hostPath: path: /var/lib/docker/containers
thanks， it works for me
escoffier
on 7 Sep 2020
Related issues
yishaihl
·
6Comments
repeatedly
·
11Comments
fadingdust
·
9Comments
jfillman
·
9Comments
artempronevskiy
·
10Comments
Most helpful comment
Instead of giving cluster-admin to the kube-system:default serviceaccount, you should modify the daemonset to use the fluentd serviceaccount created in rbtr's example. Here's a complete example for syslog: