Is this a request for help?: No
Is this a BUG REPORT or FEATURE REQUEST? (choose one): BUG REPORT
Version of Helm and Kubernetes:
Helm (Server + Client): v2.9.1
Kubernetes: v1.10.3
Which chart: stable/heapster
What happened:
Deploying the chart with rbac.create: true results in '403 Forbidden' erors in heapster log:
error while getting containers from Kubelet: failed to get all container stats from Kubelet URL "https://<Node IP>:10250/stats/container/": request failed - "403 Forbidden", response: "Forbidden (user=system:serviceaccount:kube-system:metrics-provider-heapster, verb=create, resource=nodes, subresource=stats)"
What you expected to happen:
Heapster should be allowed to create nodes/stats with RBAC.
How to reproduce it (as minimally and precisely as possible):
Configure kubelet with --read-only-port=0, enable RBAC and use /heapster --source=kubernetes:https://kubernetes.default?kubeletPort=10250&kubeletHttps=true&insecure=true&useServiceAccount=true as command.
Full values.yaml for Helm:
resizer:
enabled: false
command:
- "/heapster"
- "--source=kubernetes:https://kubernetes.default?kubeletPort=10250&kubeletHttps=true&insecure=true&useServiceAccount=true"
rbac:
create: true
Anything else we need to know:
I'm locking down Kubernetes clusters a bit by default, I'm aware this might not happen with default cluster configuration, but IMHO rbac.create should grant all RBAC permissions necessary to operate.
If this is a configuration issue in any way feel free to correct me :sweat_smile:
Thinking about it this might be related to
1.) using --authorization-mode=webhook on my kubelets
2.) the system:heapster default ClusterRole not including this permission.
Should I raise this issue with Kubernetes instead?
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Any further update will cause the issue/pull request to no longer be considered stale. Thank you for your contributions.
This issue is being automatically closed due to inactivity.
Found solution for Kubernetes 1.13.4
helm install \
--name heapster \
--namespace kube-system \
--set rbac.create=true \
--set image.repository=k8s.gcr.io/heapster-amd64 \
--set image.tag=v1.5.4 \
--set command='{/heapster,--source=kubernetes.summary_api:https://kubernetes.default?kubeletHttps=true&kubeletPort=10250&insecure=true}' \
stable/heapster
cat <<EOF | kubectl apply -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: system:heapster
rules:
- apiGroups:
- ""
resources:
- pods
- nodes
- namespaces
verbs:
- get
- list
- watch
- apiGroups:
- extensions
resources:
- deployments
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- nodes/stats
verbs:
- get
EOF
Most helpful comment
Found solution for Kubernetes 1.13.4