Operator-sdk: [ansible] Failed to create or get service for metrics: services "operator" is forbidden: cannot set blockOwnerDeletion if an ownerReference refers to a resource you can't set finalizers on

[gbaufake]

Bug Report

What did you do?
Patched an ansible operator based deployment and removed all env from the containers
(WATCH_NAMESPACE and OPERATOR_NAME)

What did you expect to see?
A container that can be destroyed.

What did you see instead? Under which circumstances?
I see two containers with two different logs
{"level":"error","ts":1563984394.5846035,"logger":"cmd","msg":"","Namespace":"","error":"OPERATOR_NAME environment variable not set","stacktrace":"github.com/operator-

kiali-operator-5d9545df59-qnhgc-operator.log

and a second one with
{"level":"info","ts":1563983817.8586397,"logger":"leader","msg":"Continuing as the leader."} {"level":"error","ts":1563983817.943182,"logger":"cmd","msg":"Exposing metrics port failed.","Namespace":"","error":"failed to create or get service for metrics: services \"kiali-operator\" is forbidden: cannot set blockOwnerDeletion if an ownerReference refers to a resource you can't set finalizers on:
kiali-operator-946947655-j5fpp-operator.log

and those containers are not being deleted with -f

Environment

• operator-sdk version: 0.8.0

• go version:go version go1.12.7 darwin/amd64

• Kubernetes version information:

• Kubernetes cluster kind: openshift 4.1
  kubernetes 1.13.4

• Are you writing your operator in ansible, helm, or go? ansible

Possible Solution

Additional context
Add any other context about the problem here.

[gbaufake]

It seems after deleting the cr and removing the finalizers, it was possible to remove the pods.

[fabianvf]

The Failed to create or get service for metrics: services "operator" is forbidden: cannot set blockOwnerDeletion ... message is usually due to not having permission to create/view/edit the deployments/finalizers and services/finalizers resources, which is required for metrics.

As for the deletion, are you deleting the deployment or the pods, and how did you deploy your operator?

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[jmazzitelli]

just fyi...

I have ansible operator 0.10.0 as my base image and I just tried it. I do NOT have services/finalizers, I just have "update" verb on deployments/finalizers and I see this in my logs (which is good):

{"level":"info","ts":1572880644.1276553,"logger":"metrics","msg":"Metrics Service object created","Service.Name":"kiali-operator-metrics","Service.Namespace":"kiali-operator"}

I then removed my operator to try a second test. I changed my role and removed this:

- apiGroups: ["apps"]
  resourceNames:
  - kiali-operator
  resources:
  - deployments/finalizers
  verbs:
  - update

I re-installed the operator and it failed with:

{"level":"error","ts":1572880854.9029043,"logger":"cmd","msg":"Exposing metrics port failed.","Namespace":"kiali-operator","error":"failed to create or get service for metrics: services \"kiali-operator-metrics\" is forbidden: cannot set blockOwnerDeletion if an ownerReference refers to a resource you can't set finalizers on: , <nil>",

So it looks like you only need "update" on "deployments/finalizers"

[openshift-bot]

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

[davidkirwan]

Just hit this same issue yesterday with operator-sdk v0.12.1

Fix was to add deployments/finalizers update permissions:

- apiGroups:
  - apps
  resources:
    - deployments/finalizers
  verbs:
    - update

[camilamacedo86]

Hi @davidkirwan,

It make sense. Really thank you for share. We will look at that.

[camilamacedo86]

Hi @davidkirwan and @gbaufake,

We could check that the issue is faced when the required permission as point out in the comments is not present in the role.ymal. It is also described already in the docs here.

Also, was verified that any project made using the latest version of SDK will have the permission:

- apiGroups:
  - apps
  resources:
  - deployments/finalizers
  resourceNames:
  - <operator-name>
  verbs:
  - "update"

In this way, I am closing this one since shows all sorted out. However, please feel free to re-open if yu see that something still required to be addressed here.

[wtrocki]

If you are using the new kubebuilder based operator try

// +kubebuilder:rbac:groups=apps,resources=deployments/finalizers,verbs=update;patch

[wtrocki]

I have the same issue but cannot manage to fix that by the line above:

2020-12-08T12:12:39.723Z    ERROR   controllers.Something Problem with creating deployment  {"": "binding/3sdgdfgfgsdfdsf-serviceapi", "error": "deployments.apps \"3sdgdfgfgsdfdsf-serviceapi-deployment\" is forbidden: cannot set blockOwnerDeletion if an ownerReference refers to a resource you can't set finalizers on: , <nil>"}

@camilamacedo86 I have noticed that you have added there resourceNames: but my generated manifest missing that. Is that required?