Lens: Non-Admin RBAC to View Metrics

[reddog335]

Describe the bug
I'm having an issue viewing metrics for a non-admin user. I can view all the metrics as cluster admin; however, when I use a developer service account with less privileges, none of the metrics appear. I have created a cluster metrics role giving get, list and watch on nodes, nodes.metrics.k8s.io and pods.nodes.metrics.k8s.io and created the necessary rolebinding for the developer service account. Are there other resources I need to give the developer service account access to in order view the metrics in Lens? I'm not seeing and RBAC DENY messages in the kube-apiserver logs. I appreciate any help you can provide. Thanks!

To Reproduce
Steps to reproduce the behavior:

1. Add a cluster to Lens using a non-admin user kubeconfig. The non-admin user has a rolebinding to a cluster metrics role giving get, list and watch on nodes, nodes.metrics.k8s.io and pods.nodes.metrics.k8s.io
2. Click on 'Workloads'
3. Click on 'Pods'
4. You will see "Metrics not available at the moment." at the top where the metrics are normally displayed.

Expected behavior
The pod metrics will be displayed at the top of the pod frame.

Screenshots

Environment (please complete the following information):

• Lens Version: 3.5.1
• OS: Windows 10 Enterprise
• Installation method (e.g. snap or AppImage in Linux): Downloaded and ran Lens-Setup-3.5.1.exe as administrator.

Logs:
When you run the application executable from command line you will see some logging output. Please paste them here:

info: Checking for update
info: Update for version 3.5.1 is not available (latest version: 3.5.1, downgrade is disallowed).
warn: PrometheusOperator: failed to list services: [object Object]
warn: PrometheusHelm: failed to list services: [object Object]
warn: PrometheusLens: failed to list services: services "prometheus" is forbidden: User "system:serviceaccount:vcs-lab2:k8-dashboard-vcs-lab2" cannot get resource "services" in API group "" in the namespace "lens-metrics"
warn: PrometheusOperator: failed to list services: [object Object]
warn: PrometheusHelm: failed to list services: [object Object]
warn: PrometheusLens: failed to list services: services "prometheus" is forbidden: User "system:serviceaccount:vcs-lab2:k8-dashboard-vcs-lab2" cannot get resource "services" in API group "" in the namespace "lens-metrics"
warn: PrometheusLens: failed to list services: services "prometheus" is forbidden: User "system:serviceaccount:vcs-lab2:k8-dashboard-vcs-lab2" cannot get resource "services" in API group "" in the namespace "lens-metrics"
warn: PrometheusHelm: failed to list services: [object Object]
warn: PrometheusOperator: failed to list services: [object Object]
info: using lens as prometheus provider
warn: PrometheusLens: failed to list services: services "prometheus" is forbidden: User "system:serviceaccount:vcs-lab2:k8-dashboard-vcs-lab2" cannot get resource "services" in API group "" in the namespace "lens-metrics"
warn: PrometheusHelm: failed to list services: [object Object]
warn: PrometheusOperator: failed to list services: [object Object]
info: using lens as prometheus provider
warn: watch ended: Error: Forbidden
warn: watch ended: Error: Forbidden
warn: watch ended: Error: Forbidden
warn: watch ended: Error: Forbidden
warn: watch ended: Error: Forbidden
warn: watch ended: Error: Forbidden
Your logs go here...

Kubeconfig:
Quite often the problems are caused by malformed kubeconfig which the application tries to load. Please share your kubeconfig, remember to remove any secret and sensitive information.

apiVersion: v1
kind: Config
clusters:
- name: k8-tkg-lab2
  cluster:
    certificate-authority-data: <redacted>
    server: https://k8-tkg-lab2.<redacted>:6443
contexts:
- name: k8-dashboard-vcs-lab2
  context:
    cluster: k8-tkg-lab2
    namespace: vcs-lab2
    user: k8-dashboard-vcs-lab2
current-context: k8-dashboard-vcs-lab2
users:
- name: k8-dashboard-vcs-lab2
  user:
    token: <redacted>

Additional context
Add any other context about the problem here.

[pyrkamarcin]

I love Lens, it is fantastic. I'm having an issue viewing metrics for a non-admin user. I can view all the metrics as cluster admin; however, when I use a developer service account with less privileges, none of the metrics appear. I have created a cluster metrics role giving get, list and watch on nodes, nodes.metrics.k8s.io and pods.nodes.metrics.k8s.io and created the necessary rolebinding for the developer service account. Are there other resources I need to give the developer service account access to in order view the metrics in Lens? I'm not seeing and RBAC DENY messages in the kube-apiserver logs. I appreciate any help you can provide. Thanks!

Can you adapt your issue to general schema, like: https://github.com/lensapp/lens/issues/new?assignees=&labels=&template=bug_report.md&title=

[reddog335]

Adapted, apologies for the poor form.

[just1689]

@reddog335 I have the same issue. I'm creating service accounts with cluster role bindings with get list and watch on just about everything. Could I ask you have found a way to make metrics views work with a read only user?

I've also try launching Lens from command line in the hopes that an error message would inform me which resource / etc I was missing.

[just1689]

Greetings

Haven't figured out exactly what it needs but I'm giving my read only users this ClusterRole for now:

https://gist.github.com/just1689/272f3e78a540c10b7bda390eef9b4473

My plan is to cut it down as I figure out what is the minimum they need.

It does work with this though.

[just1689]

I've made it more explicit now - secrets can't be read

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: read-only-clusterrole
rules:
  - nonResourceURLs:
      - /metrics
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - bindings
      - componentstatuses
      - configmaps
      - endpoints
      - events
      - limitranges
      - namespaces
      - namespaces/finalize
      - namespaces/status
      - nodes
      - nodes/proxy
      - nodes/status
      - persistentvolumeclaims
      - persistentvolumeclaims/status
      - persistentvolumes
      - persistentvolumes/status
      - pods
      - pods/attach
      - pods/binding
      - pods/eviction
      - pods/exec
      - pods/log
      - pods/proxy
      - pods/status
      - podtemplates
      - replicationcontrollers
      - replicationcontrollers/scale
      - replicationcontrollers/status
      - resourcequotas
      - resourcequotas/status
      - serviceaccounts
      - services
      - services/proxy
      - services/status
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - apps
    resources:
      - controllerrevisions
      - daemonsets
      - daemonsets/status
      - deployments
      - deployments/scale
      - deployments/status
      - replicasets
      - replicasets/scale
      - replicasets/status
      - statefulsets
      - statefulsets/scale
      - statefulsets/status
    verbs:
      - list
      - get
      - watch
  - apiGroups:
      - batch
    resources:
      - jobs
      - jobs/status
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - autoscaling
    resources:
      - horizontalpodautoscalers
      - horizontalpodautoscalers/status
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - storage.k8s.io
    resources:
      - csidrivers
      - csinodes
      - storageclasses
      - volumeattachments
      - volumeattachments/status
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - networking.k8s.io
    resources:
      - networkpolicies
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - scheduling.k8s.io
    resources:
      - priorityclasses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - node.k8s.io
    resources:
      - runtimeclasses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses
      - ingresses/status
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - events.k8s.io
    resources:
      - events
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - apiextensions.k8s.io
    resources:
      - customresourcedefinitions
      - customresourcedefinitions/status
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - apiregistration.k8s.io
    resources:
      - apiservices
      - apiservices/status
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - discovery.k8s.io
    resources:
      - endpointslices
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - metrics.k8s.io
    resources:
      - pods
      - nodes
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - policy
    resources:
      - poddisruptionbudgets
      - poddisruptionbudgets/status
      - podsecuritypolicies
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - rbac.authorization.k8s.io
    resources:
      - clusterrolebindings
      - clusterroles
      - rolebindings
      - roles
    verbs:
      - get
      - list
      - watch

[reddog335]

Thanks @just1689 I'll give this a try.

[reddog335]

That worked like a champ @just1689, thank you very much! I removed the list verb and created a rolebinding in the lens-metrics namespace for each individual application namespace.