Brigade: [proposal] Remove secrets from project's configuration into separate secrets

[danmx]

It would be easier to manage secrets if project's configuration would only have references to particular secrets.

E.g. this could allow jobs running in different namespaces to directly access secrets deployed there.

[krancour]

This might possibly be rendered moot by #1035, which suggests that a Kubernetes secret is, perhaps, not the best way to store project configuration. I'm speculating about implementation details here, but in the future, one-off secrets might be derived from some other source and then destroyed when they are no longer needed.

this could allow jobs running in different namespaces to directly access secrets deployed there.

Are you somehow managing to run jobs in different namespaces? Brigade doesn't currently support that.

[danmx]

I was thinking that running jobs in project specific namespaces could be a good security feature

[krancour]

It's something that was considered at one point and never implemented. If you'd like to open a new issue specifically about that, I think that's something we'd entertain for 2.0. It makes a lot of sense-- and not only for security reasons. Personally, I like the idea of things happening in team / project specific namespaces wherein a cluster administrator _could_ set set resource quotas.

[mmlac]

@danmx Each project having its own namespace is a core requirement for us to even consider running multi-tenant builds on Brigade. Imho Kubernetes secrets should not be replaced with a different set of secrets, as K8s already provides solid RBAC controls around them.
Can you reference the issue here / tag me in the new one for the namespaces proposal?

[krancour]

Imho Kubernetes secrets should not be replaced with a different set of secrets, as K8s already provides solid RBAC controls around them.

See #1034. We are quite possibly looking toward a future where k8s is a mere implementation detail of some driver/execution layer. Necessarily, that would force us consider how concepts like users, roles, projects, secrets, etc. can be decoupled from k8s.

fwiw, if we get the architecture right, I don't see multi-tenancy as necessarily being too difficult to also achieve. I really like the idea that projects or teams could be scoped not only to their own namespace, but even, potentially (optionally) to their own cluster.

[danmx]

@mmlac I’m not suggesting dropping Kubernetes secrets as a secrets storage. My point is to decouple secrets like api keys etc. from the configuration and put them in separate Kubernetes Secrets. Therefore we can use regular ConfigMap for configuration and only reference Kubernetes Secrets which will simplify secrets management (e.g. a single secret rotation).

For example: gateways’ secrets can be placed in brigade namespace where they are controlled by service admins but projects’ secrets are placed in corresponding namespaces where users can freely modify them.

I think https://github.com/brigadecore/brigade/issues/755 issue is covering running in multiple namespaces.

[krancour]

Per the 2.0 proposal, which has been ratified and is now guiding the 2.0 development effort, secret management, like the rest of Brigade, is getting a significant overhaul. Since this is well covered by the proposal, it probably doesn't make sense to track this as a discrete issue anymore.