External-dns: Unknown Serviceaccount error

[ekoome]

I have used the RBAC manifest for Digital Ocean, but get the following error when deployed to GKE.

external-dns | 1 Jun 2018, 14:36:27 | time="2018-06-01T13:36:27Z" level=error msg="services is forbidden: User \"system:serviceaccount:default:default\" cannot list services at the cluster scope: Unknown user \"system:serviceaccount:default:default\""
-- | -- | --

Manifest

`apiVersion: v1
kind: ServiceAccount
metadata:

name: external-dns

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: external-dns
rules:

• apiGroups: [""]
  resources: ["services"]
  verbs: ["get","watch","list"]
• apiGroups: [""]
  resources: ["pods"]
  verbs: ["get","watch","list"]
• apiGroups: ["extensions"]
  resources: ["ingresses"]

verbs: ["get","watch","list"]

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: external-dns-viewer
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: external-dns
subjects:

• kind: ServiceAccount
  name: external-dns

namespace: default

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: external-dns
spec:
strategy:
type: Recreate
template:
metadata:
labels:
app: external-dns
spec:
containers:
- name: external-dns
image: registry.opensource.zalan.do/teapot/external-dns:v0.5.1
args:
- --source=service # ingress is also possible
- --domain-filter=xxx.mydomain.com # (optional) limit to only example.com domains; change to match the zone created above.
- --provider=digitalocean
env:
- name: DO_TOKEN
value: "KEY"`

What do I need to do to make it work?

[hjacobs]

You are not assigning the created service account to your pod, see https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/

[ekoome]

Added serviceaccount into pod under spec. That is serviceAccountName: external-dns. Probably need to add this in the RBAC sample manifest.

[hjacobs]

@ekoome where do you see it missing? I checked the docs and the serviceAccountName is there in the RBAC sample manifest: https://github.com/kubernetes-incubator/external-dns/blob/a104993630f9731ab59a4b43bcfd755577c324b8/docs/tutorials/digitalocean.md#manifest-for-clusters-with-rbac-enabled

If it's missing in some other sample manifest, please let us know.

[jeffdeville]

@hjacobs - We're running into this error:

time="2018-06-18T13:24:49Z" level=error msg="nodes is forbidden: User "system:serviceaccount:default:external-dns" cannot list nodes at the cluster scope: Unknown user "system:serviceaccount:default:external-dns"

Our yaml:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: external-dns
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: external-dns
rules:
  - apiGroups: [""]
    resources: ["services"]
    verbs: ["get","watch","list"]
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get","watch","list"]
  - apiGroups: ["extensions"]
    resources: ["ingresses"]
    verbs: ["get","watch","list"]
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: external-dns-viewer
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: external-dns
subjects:
  - kind: ServiceAccount
    name: external-dns
    namespace: kube-system
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: external-dns
  namespace: kube-system
spec:
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: external-dns
    spec:
      serviceAccountName: external-dns
      containers:
        - name: external-dns
          image: registry.opensource.zalan.do/teapot/external-dns:v0.5.3
          args:
            - --source=service
            - --provider=google
            - --registry=txt
            - --txt-owner-id=my-identifier

[jeffdeville]

FWIW, I tried to use the helm package as well, with the same result:

helm upgrade --install external-dns stable/external-dns \
                         --set google.project=PROJECT \
                         --set image.tag=v0.5.3 \
                         --set logLevel=debug \
                         --set policy=sync \
                         --set rbac.create=true \
                         --set provider=google

Cluster Version: 1.10.2-gke.3

[rdeknijf]

The log message is misleading. I've noticed this before with RBAC: the account is there, it just doesn't have the correct permissions. In this case it requires the following additional ones:

- apiGroups: [""]
  resources: ["nodes"]
  verbs: ["get","watch","list"]

[linki]

addressed in #604

[jeffdeville]

Thanks @rdeknijf !

[andreasolund]

Would it be possible to get the official chart updated with the latest changes? Others might be hitting this wall when missing the nodes list permission https://github.com/kubernetes/charts/pull/6176

[njuettner]

@andreasolund I wrote a comment that 0.5.4 is available now

[ilmatic]

@njuettner In terms of the helm chart, is this just a matter of updating image.tag in the values file to "v0.5.4"? I tried updating the values yaml with that and am still getting the services is forbidden error

[andreasolund]

@ilmatic No, if you're using RBAC, you need the nodes resource permission. Which is being updated in https://github.com/kubernetes/charts/pull/6176. You can use v0.5.2 until the new chart has been released, or download and use a local copy of the chart with the new changes.

[njuettner]

@ilmatic As @andreasolund mentioned, we updated the RBAC permissions in each tutorial https://github.com/kubernetes-incubator/external-dns/tree/master/docs/tutorials

[flyboarder]

I am still running into this issue using the new permissions and v0.5.4

[blakestoddard]

This is still present when using this RBAC: https://github.com/kubernetes-incubator/external-dns/blob/master/docs/tutorials/gke.md#manifest-for-clusters-with-rbac-enabled

[flyboarder]

I have a working cluster using:

- apiGroups: [""]
  resources: ["nodes"]
  verbs: ["get","watch","list"]