Operator-sdk: Cluster scoped operator support

[philhug]

sdk.Watch currently requires you to provide a namespace.
Are there plans to support cluster scoped CRDs? Or is this out of scope for operator-sdk?

[philips]

The opinion that we have developed overtime is that installing Operators in each namespace is the best choice as it enables decoupled upgrades, namespace isolation for failure/monitoring, and differing API definitions.

By using Operator Lifecycle Management the intention is to simplify the management of these per-namespace Operators.

cc @jzelinskie

[ichekrygin]

Perhaps it is a misunderstanding on my part, but the way I interpret this issue is the operator-sdk does not facilitate functionality to define/operate on CRD's at the cluster scope, i.e. resources like namespaces, cluster-roles, etc., and not which namespace the operator itself is deployed into.

For example, something like this: https://github.com/kubernetes-sigs/kubebuilder/blob/master/docs/adding_non_namespaced_resources.md

[ichekrygin]

To follow up on this, @philips @jzelinskie, is there a way currently or are there any plans to enable non-namespaced resources support in the operator-sdk?

[jzelinskie]

For the SDK, this requires some discussion because we need to not only consider the code changes to the Operator SDK, but the architectural changes to the Operator Framework as a whole.

Right now, the Operator Lifecycle Manager is designed to be flexible enough to run entirely namespaced or at the cluster scope. This is extremely important, because it enables developers with only permission to create resources in their namespace the ability to leverage the Operator Framework.

However, there is another use case of the cluster administrator providing cluster-wide services or services that require cluster-wide permissions. Today, with some tweaking of the deployment configuration, you can install the Operator Lifecycle Manager in such a way that Operators are only installed and deployed into one namespace that is locked down by RBAC and runs Operators with privileged access to the cluster. You need this isolated namespace in order to avoid privilege escalation from other users of the Kubernetes cluster.

In addition to whatever code we add to the SDK to help write Operators that fall into the latter use case, we also need to do some work to identify which use case an Operator falls into, and organize those Operators into separate catalogs.

[kfox1111]

don't you need extra cluster wide privilege anyway though to register the crds? I didn't think there was a namespace specific version of CustomResourceDefinition yet?

[jzelinskie]

don't you need extra cluster wide privilege anyway though to register the crds?

This is true, and all of that functionality of OLM is isolated to the optional _catalog_ operator. If you'd like to leverage automated updates, you need something with the permission to install requirements. However, you can still satisfy the requirements for an Operator manually and apply a ClusterServiceVersion-v1 to a namespace and have the _olm_ operator properly manage the lifecycle of the Operator. Think of this like side-loading an APK on an Android phone or using dpkg -i on Debian.

[kfox1111]

Is there any effort going on in the k8s community to have a namespacedCustomResourceDefinition so that operators installed into a namespace could version their CRD at the namespace level and would only need namespaced permissions to create them? I could see that being very valuable for testing purposes and for multitenant situations.

[joelanford]

As of v0.1.0, operators can now watch resources across the cluster by leaving namespace unset (or set to an empty string) when creating the controller-runtime manager:

mgr, err := manager.New(cfg, manager.Options{})

And as of v0.2.0, it is possible to scaffold a new cluster-scoped operator with operator-sdk new --cluster-scoped.

Closing this issue since the original problem has been solved. Please open a new ticket if more discussion is necessary.