Operator-sdk: Failed with "the server could not find the requested resource"

[ladmitry]

Bug Report

What did we do?
Trying to deploy custom resource via the operator:
Facing the issue: _the server could not find the requested resource_

What we expect:
Below is the stdout log from the operator deployed on Openshift 3.11:
2019-02-21T13:55:19.173Z INFO logging_event_handler [playbook task] {"name": "example-full", "namespace": "redis-cluster", "gvk": "db.mysite.com/v1alpha1, Kind=RedisCluster", "event_type": "playbook_on_task_start", "job": "3033838985510594553", "EventData.Name": "Gathering Facts"}

What we see instead:
The same CRD, the same role, service account, permissions etc. on Openshift 3.10:
{"level":"error","ts":1550756524.2727947,"logger":"kubebuilder.controller","msg":"Reconciler error","controller":"rediscluster-controller","request":"redis-cluster/example-full","error":"the server could not find the requested resource","stacktrace":"github.com/operator-framework/operator-sdk/vendor/github.com/go-logr/zapr.(*zapLogger).Error\n\t/home/joe/go/src/github.com/operator-framework/operator-sdk/vendor/github.com/go-logr/zapr/zapr.go:128\ngithub.com/operator-framework/operator-sdk/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/joe/go/src/github.com/operator-framework/operator-sdk/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:215\ngithub.com/operator-framework/operator-sdk/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\t/home/joe/go/src/github.com/operator-framework/operator-sdk/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:158\ngithub.com/operator-framework/operator-sdk/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/home/joe/go/src/github.com/operator-framework/operator-sdk/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133\ngithub.com/operator-framework/operator-sdk/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/home/joe/go/src/github.com/operator-framework/operator-sdk/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:134\ngithub.com/operator-framework/operator-sdk/vendor/k8s.io/apimachinery/pkg/util/wait.Until\n\t/home/joe/go/src/github.com/operator-framework/operator-sdk/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88"}

Environment

• operator-sdk version: 0.3.0, 0.4.1, 0.5.0
• openshift v3.10.0+fd501dd-48 *
• kubernetes v1.10.0+b81c8f8 *
• operator code is written on: Ansible *

[shawn-hurley]

Can you please paste your CRD here.

The ansible operator needs to use the status subresource which requires k8s 1.10 or higher, you get the same error when the status subresource does not exist, and I am pretty sure this is the problem.

Please note this from the README

Access to a kubernetes v.1.11.0+ cluster.

[ladmitry]

Sure, here is CRD:

apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: redisclusters.db.mysite.com
spec:
  group: db.mysite.com
  names:
    kind: RedisCluster
    listKind: RedisClusterList
    plural: redisclusters
    singular: rediscluster
    shortNames:
    - rcl
  scope: Namespaced
  version: v1alpha1
  subresources:
    status: {}
  validation:
    openAPIV3Schema:
      properties:
        spec:
          type: object

[shawn-hurley]

@ladmitry I think that 3.11 does this for you so for 3.10 I think you need to do this extra step: https://docs.openshift.com/container-platform/3.10/admin_guide/custom_resource_definitions.html#creating-aggregated-cluster-role-crd_admin-guide-custom-resources

[ladmitry]

Unfortunately, it doesn't work.
We've created the ClusterRoles, but the error is still in place and the cause is somewhere else.
Below is the ClusterRole we created. Besides, we are namespace-scoped, not cluster-scoped. So it seems it's not a solution.

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: aggregate-redisclusters-admin-edit
  labels:
    rbac.authorization.k8s.io/aggregate-to-admin: "true"
    rbac.authorization.k8s.io/aggregate-to-edit: "true"
rules:
- apiGroups: ["db.mysite.com"]
  resources: ["redisclusters"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete", "deletecollection"]
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: aggregate-redisclusters-view
  labels:
    rbac.authorization.k8s.io/aggregate-to-view: "true"
rules:
- apiGroups: ["db.mysite.com"]
  resources: ["redisclusters"]
  verbs: ["get", "list", "watch"]

[shawn-hurley]

I think you need to grant access to the subresources as well

[darrylb-github]

I get the same error when binding the service account to the cluster-admin ClusterRole

[shawn-hurley]

@ladmitry @darrylb-github Any chance that you could make a small reproducer that I could run?

[chancez]

I don't think the update subresource is enabled for CRDs in 3.10

[ashokhein]

I am facing the same issue when I followed this link https://github.com/operator-framework/operator-sdk/blob/master/doc/ansible/user-guide.md

operator-sdk version: v0.7.0
Kubectl Client Version: v1.9.11
K8s Server Version: v1.9.11
ansible 2.7.9

I tried to run both inside and outside the cluster as per README.md and receiving the same errors:

{"level":"error","ts":1556555179.8817847,"logger":"reconciler","msg":"Unable to update the status to mark cr as running","job":"1730092421274156201","name":"example-foo","namespace":"default","error":"the server could not find the requested resource","stacktrace":"github.com/operator-framework/operator-sdk/vendor/github.com/go-logr/zapr.(*zapLogger).Error\n\tsrc/github.com/operator-framework/operator-sdk/vendor/github.com/go-logr/zapr/zapr.go:128\ngithub.com/operator-framework/operator-sdk/pkg/ansible/controller.(*AnsibleOperatorReconciler).Reconcile\n\tsrc/github.com/operator-framework/operator-sdk/pkg/ansible/controller/reconcile.go:126\ngithub.com/operator-framework/operator-sdk/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsrc/github.com/operator-framework/operator-sdk/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:215\ngithub.com/operator-framework/operator-sdk/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\tsrc/github.com/operator-framework/operator-sdk/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:158\ngithub.com/operator-framework/operator-sdk/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\tsrc/github.com/operator-framework/operator-sdk/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133\ngithub.com/operator-framework/operator-sdk/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil\n\tsrc/github.com/operator-framework/operator-sdk/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:134\ngithub.com/operator-framework/operator-sdk/vendor/k8s.io/apimachinery/pkg/util/wait.Until\n\tsrc/github.com/operator-framework/operator-sdk/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88"}

[shawn-hurley]

Hello @ashokhein

I see that you are using k8s 1.9 which we do not support.

Please note this from the README

Access to a kubernetes v.1.11.0+ cluster.

Thanks,

Shawn

[shawn-hurley]

@darrylb-github @ladmitry was there a reproducer that I could run to test this out?

[vhosakot]

I saw the same error with kubebuilder version 2.0.0-alpha.1:

"error": "the server could not find the requested resource (put cronjobs.batch.sun.kubebuilder.io cronjob-sample)"}

Adding the // +kubebuilder:subresource:status comment above the CronJob struct in cronjob_types.go, and then running make manifests per https://book-v1.book.kubebuilder.io/basics/status_subresource.html resolved the error for me:

// +kubebuilder:subresource:status
type CronJob struct {

make manifests

status subresource in CRD after running make manifests:

spec:
...
  subresources:
    status: {}
...

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[ahrtr]

Reproduced this issue following the https://docs.okd.io/latest/operators/osdk-getting-started.html

[openshift-bot]

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

[openshift-bot]

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

[openshift-ci-robot]

@openshift-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.