Dashboard: Limit dashboard visibility to a single namespace

Created on 6 Feb 2020 · 33Comments · Source: tektoncd/dashboard

We have different teams and want give them different dashboard to manage their own pipelines.
Basically we are giving each team a namespace and deploy tekton pipeline and dashboard on it.
K8S RBAC provides isolation so team A cannot see team B pipelines.
The problem is that tekton dashboard needs cluster roles to list the pipelines in all namespaces.
We have tried to change the tekton-dashboard-minimal ClusterRole/ClusterRoleBinding to Role/RoleBinding, but that makes the dashboard failing to start.

Expected behavior

Tekton dashboard should work with only Role/RoleBinding and only show pipelines in its own namespace

Actual behavior

Tekton dashboard is failing because it needs Tekton ClusterRole/ClusterRoleBinding to list all pipelines in the cluster

Steps to reproduce the problem

Change Dashboard ClusterRole & ClusterRoleBinding to Role & RoleBinding
Deploy the dashboard
Dashboard pod fails to start

Environment

Kubernetes Platform:
GKE v1.14.10-gke.17
Tekton Pipelines version:
v0.10.0
Tekton Triggers version:
latest
Tekton Dashboard version:
latest

kinfeature lifecyclfrozen

Source

MohamedTalhaoui

👍10

All 33 comments

Yeah this is an interesting one, agree it'd be nice to have - so can you share the pod log details please or the errors you're seeing?

I wonder if in our UI we do an all namespaces lookup first and you're getting a displayed RBAC error on the page 🤔

a-roberts on 6 Feb 2020

Thanks for the prompt reply.
I think your assumption is correct.

here are the logs I get:

E0206 16:03:06.958071       1 reflector.go:134] runtime/asm_amd64.s:1357: Failed to list *v1alpha1.ClusterTask: clustertasks.tekton.dev is forbidden: User "system:serviceaccount:team-a:tekton-dashboard" cannot list resource "clustertasks" in API group "tekton.dev" at the cluster scope
E0206 16:03:07.445879       1 reflector.go:134] runtime/asm_amd64.s:1357: Failed to list *v1alpha1.PipelineResource: pipelineresources.tekton.dev is forbidden: User "system:serviceaccount:team-a:tekton-dashboard" cannot list resource "pipelineresources" in API group "tekton.dev" at the cluster scope
E0206 16:03:07.751665       1 reflector.go:134] runtime/asm_amd64.s:1357: Failed to list *v1.Secret: secrets is forbidden: User "system:serviceaccount:team-a:tekton-dashboard" cannot list resource "secrets" in API group "" at the cluster scope
E0206 16:03:07.961594       1 reflector.go:134] runtime/asm_amd64.s:1357: Failed to list *v1alpha1.ClusterTask: clustertasks.tekton.dev is forbidden: User "system:serviceaccount:team-a:tekton-dashboard" cannot list resource "clustertasks" in API group "tekton.dev" at the cluster scope
E0206 16:03:08.448351       1 reflector.go:134] runtime/asm_amd64.s:1357: Failed to list *v1alpha1.PipelineResource: pipelineresources.tekton.dev is forbidden: User "system:serviceaccount:team-a:tekton-dashboard" cannot list resource "pipelineresources" in API group "tekton.dev" at the cluster scope
E0206 16:03:08.753924       1 reflector.go:134] runtime/asm_amd64.s:1357: Failed to list *v1.Secret: secrets is forbidden: User "system:serviceaccount:team-a:tekton-dashboard" cannot list resource "secrets" in API group "" at the cluster scope
E0206 16:03:08.972151       1 reflector.go:134] runtime/asm_amd64.s:1357: Failed to list *v1alpha1.ClusterTask: clustertasks.tekton.dev is forbidden: User "system:serviceaccount:team-a:tekton-dashboard" cannot list resource "clustertasks" in API group "tekton.dev" at the cluster scope
E0206 16:03:09.450402       1 reflector.go:134] runtime/asm_amd64.s:1357: Failed to list *v1alpha1.PipelineResource: pipelineresources.tekton.dev is forbidden: User "system:serviceaccount:team-a:tekton-dashboard" cannot list resource "pipelineresources" in API group "tekton.dev" at the cluster scope

MohamedTalhaoui on 6 Feb 2020

Any update or progress here ?

MohamedTalhaoui on 26 Mar 2020

Any update or progress here ?

Hey @MohamedTalhaoui, sorry for taking so long to get back to you - nobody's working on this but still think it's a good idea! Not sure if you can see it but we have a ZenHub board so I've put it up the backlog as it's an interesting piece of work anyone could pick up (and most of the active developers on the Dashboard here at IBM look at said board when looking for something new to pick up).

For anyone who does pick this up, do a good analysis and explore what options we have.

a-roberts on 2 Apr 2020

Hi,
thanks for the update.
Unfortunately I cannot see the ZenHub board but looking forward for more updates

MohamedTalhaoui on 7 Apr 2020

Hi,
any update ?

MohamedTalhaoui on 8 May 2020

I made a step in this direction here: https://github.com/tektoncd/experimental/pull/536

Pipelines don't need to be deployed in each namespace, a single instance of pipelines should be ok.
On the other hand, dashboard would need to be deployed multiple times.

It's probably not technically difficult to implement but how to make this available to the end user ?

eddycharly on 11 May 2020

Hey @MohamedTalhaoui and @eddycharly, I brought this up on the working group call today - it's something we should call out as a feature to add to our roadmap I think.

I agree, it'd need to be a "can have multiple installs of the Dashboard" implementation, so a good first step would probably be to have Roles and RoleBindings everywhere as a version (ideally through kustomize), or we somehow lay down those roles programatically based on an installation mode. I don't really want to see us having an increased test matrix (so, read-only * openshift or not * multiple install namespace mode or not), so design contributions would be awesome here

@AlanGreene ^^

a-roberts on 11 May 2020

@a-roberts yep I heard that ;)
We can still use ClusterRole but use RoleBinding instead of ClusterRoleBinding.

eddycharly on 11 May 2020

😄1

I wouldn’t create the roles programmatically, it would require the backend to have permissions to create roles, I would prefer to avoid that for security reasons.

eddycharly on 11 May 2020

I wouldn’t create the roles programmatically, it would require the backend to have permissions to create roles, I would prefer to avoid that for security reasons.

@a-roberts yep I heard that ;)
We can still use ClusterRole but use RoleBinding instead of ClusterRoleBinding.

Ah, didn't realise you were listening in! Doesn't ClusterRole require cluster-admin permissions to install? Happy to be mistaken and that would simplify things.

I'm wondering as this was brought up here (definitely a related issue): and it'd be great if addressing this issue removes any cluster admin necessities. +1 on avoiding doing things programatically.

a-roberts on 11 May 2020

Doesn't ClusterRole require cluster-admin permissions to install?

I think it's like other resources, RBAC decides if one is allowed or not to operate ClusterRoles.

Anyway i don't think the requirement here is to let users install the dashboard themself, it would probably be installed by someone in the ops team who has the necessary permissions to do so.

But once installed, it should show resources from a single namespace etc... not the whole cluster.

I could have misunderstood the issue though (@MohamedTalhaoui ?)

eddycharly on 11 May 2020

Issue https://github.com/tektoncd/dashboard/issues/1355 is not related, it's a breaking change in Deployment label selectors (those are immutable).

I wrote it in the description of the PR (https://github.com/tektoncd/dashboard/pull/1327)

This will introduce a breaking change when trying to update deployments as label selectors have changed.

But AFAIK this was not released yet, the user is probably trying to install master.

eddycharly on 11 May 2020

Issue #1355 is not related, it's a breaking change in Deployment label selectors (those are immutable).

I wrote it in the description of the PR (#1327)

This will introduce a breaking change when trying to update deployments as label selectors have changed.

But AFAIK this was not released yet, the user is probably trying to install master.

I see, yeah the reason I figured it's related is that they, by the sounds of it, don't have permissions to install the Dashboard in their cluster, but they do have permissions to install things into their own namespace, and so by implementing this feature, if we went down the On the other hand, dashboard would need to be deployed multiple times. route) then they'd benefit.

But indeed, everything we have is scoped to a namespace anyway, so they should be able to install into a particular namespace. Thanks for the feedback on this one and good to get the discussion going.

Yeah, But once installed, it should show resources from a single namespace etc... not the whole cluster. makes sense to me for the scope of this, let's see what @MohamedTalhaoui thinks

a-roberts on 11 May 2020

https://github.com/tektoncd/dashboard/issues/1355 is related, there are 2 issues discussed:

1 is a known breaking change related to recently added labels, the user has resolved this issue
2 is related to providing non-admin users access to the dashboard without giving them cluster-wide access

AlanGreene on 11 May 2020

Doesn't ClusterRole require cluster-admin permissions to install?

I think it's like other resources, RBAC decides if one is allowed or not to operate ClusterRoles.

Anyway i don't think the requirement here is to let users install the dashboard themself, it would probably be installed by someone in the ops team who has the necessary permissions to do so.

But once installed, it should show resources from a single namespace etc... not the whole cluster.

I could have misunderstood the issue though (@MohamedTalhaoui ?)

Exactely.
The goal is the ops provision the dashboards (1 per namespace, 1 namespace per team).
Then each team have access to its own dashboard showing its own pipelines.

MohamedTalhaoui on 11 May 2020

I guess we’re talking about strong isolation here, not just ui presentation, otherwise the drop down in the side nav would be enough 😄

Such a setup will probably require admin rights at some points, chances are that it will require to configure an ingress with authentication, register dns records, ssl certificates...

eddycharly on 11 May 2020

2 is related to providing non-admin users access to the dashboard without giving them cluster-wide access

Do you mean preventing them from creating resources in any namespace of the cluster ?

eddycharly on 11 May 2020

The goal is the ops provision the dashboards (1 per namespace, 1 namespace per team).
Then each team have access to its own dashboard showing its own pipelines.

Ok so install is not an issue, Ops team has admin privileges in the cluster right ?

If that's correct, the only thing left is limiting the actions scope of the dashboard backend to a single namespace.

eddycharly on 11 May 2020

I confirm Install is not an issue for my use case.

MohamedTalhaoui on 11 May 2020

There are still some details to be worked on but it should be pretty straight forward.

My recommended setup would be as follows:

create the tekton-pipelines namespace and install pipelines in it (the same instance will be shared by all teams, pipelines doesn’t support multi tenancy yet)
create the tekton-dashboards namespace and install dashboards in it (one dashboard instance per team but all in the same namespace)
create one namespace per team (team1, team2, etc...) to isolate resources per team (don’t necessarily grant them access to the namespace, especially if secrets and service accounts are managed by the ops team)
bind each dashboard instance to a specific namespace

With this setup a specific dashboard instance would only access a specific namespace and therefore only the pipeline runs, secrets, service accounts etc of the team associated with the namespace.

The ingress/authentication part is out of scope here, it will depend on your setup/provider.

I have other ideas to make things more flexible and potentially cross namespaces/cross teams but we’re not quite there yet so the setup above would be more easily reachable for now.

eddycharly on 12 May 2020

👍1

How would you bind a dashboard instance to a specific namespace ?
Also, to have strong isolation we are also deploying different EventListerner for each team to listen to git webhook, so a git push in team A repo does not trigger a pipeline on team B. What would be your advice here ?

MohamedTalhaoui on 12 May 2020

How would you bind a dashboard instance to a specific namespace ?

Something along those lines dashboard --tenant-namespace team1, it's not currently supported but it's the idea.

Regarding your tiggers question i don't know triggers well enough, sorry.

eddycharly on 12 May 2020

I began working on this, splitting RBAC for now.
/assign

eddycharly on 12 May 2020

❤1

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Send feedback to tektoncd/plumbing.