Kubectl: Selector does not work for service.

[J-Siu]

Is this a request for help? (If yes, you should use our troubleshooting guide and community support channels, see http://kubernetes.io/docs/troubleshooting/.):

What keywords did you search in Kubernetes issues before filing this one? (If you have found any duplicates, you should instead reply there.):

---

Is this a BUG REPORT or FEATURE REQUEST? (choose one): BUG REPORT

Kubernetes version (use kubectl version):

Client Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.3", GitCommit:"a4529464e4629c21224b3d52edfe0ea91b072862", GitTreeState:"clean", BuildDate:"2018-09-09T18:02:47Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.3", GitCommit:"a4529464e4629c21224b3d52edfe0ea91b072862", GitTreeState:"clean", BuildDate:"2018-09-09T17:53:03Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"}

Environment:

• Cloud provider or hardware configuration: VirtualBox
• OS (e.g. from /etc/os-release): Ubuntu 18.04.1 LTS
• Kernel (e.g. uname -a): Linux u64s01 4.15.0-34-generic #37-Ubuntu SMP Mon Aug 27 15:21:48 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
• Install tools: apt

apt-get update && apt-get install -y apt-transport-https curl
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb http://apt.kubernetes.io/ kubernetes-xenial main
EOF
apt-get update
apt-get install -y kubelet kubeadm kubectl
apt-mark hold kubelet kubeadm kubectl

• Others:

What happened:

Selector does not work for service.

What you expected to happen:

Selector should work for service.

How to reproduce it (as minimally and precisely as possible):

1. Deploy ghost:

kubectl run ghost --image=ghost --port 2368 --expose=true

1. Verify service:

$ kubectl get svc -o wide
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE       SELECTOR
ghost        NodePort    10.100.72.171   <none>        2368:30538/TCP   1d        run=ghost
kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP          1d        <none>

1. Try with selector:

$ kubectl get svc -o wide -l run=ghost
No resources found.

$ kubectl delete svc -l run=ghost
No resources found

Anything else we need to know:

It work if I select k8s-app

$ kubectl get svc -o wide --all-namespaces
NAMESPACE     NAME            TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE       SELECTOR
default       ghost           NodePort    10.100.72.171   <none>        2368:30538/TCP   1d        run=ghost
default       kubernetes      ClusterIP   10.96.0.1       <none>        443/TCP          1d        <none>
kube-system   kube-dns        ClusterIP   10.96.0.10      <none>        53/UDP,53/TCP    1d        k8s-app=kube-dns
kube-system   tiller-deploy   ClusterIP   10.104.224.90   <none>        44134/TCP        9h        app=helm,name=tiller

$ kubectl get svc -o wide --all-namespaces -l k8s-app=kube-dns
NAMESPACE     NAME       TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)         AGE       SELECTOR
kube-system   kube-dns   ClusterIP   10.96.0.10   <none>        53/UDP,53/TCP   1d        k8s-app=kube-dns

$ kubectl get svc -o wide --all-namespaces -l run=ghost
No resources found.

[seans3]

/kind bug
/sig cli
/area kubectl
/priority P1

[m1kola]

I can reproduce it. It's wired. When I add a new label to the service, I can select the service using this label:

kubectl label svc ghost app=something

kubectl get svc -o wide
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE       SELECTOR
ghost        ClusterIP   10.100.76.186   <none>        2368/TCP   14m       run=ghost
kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP    54d       <none>

kubectl get svc -o wide -l app=something
NAME      TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE       SELECTOR
ghost     ClusterIP   10.100.76.186   <none>        2368/TCP   14m       run=ghost

but run=ghost still doesn't return any results

kubectl get svc -o wide -l run=ghost
No resources found.

I'll look, if I can find the cause and if it.

[m1kola]

I think, it's not a bug. I just realised that the steps to reproduce do not have a step where we mark a service with a label. I didn't manage to find evidence that it's expected from kubectl run to implicitly add labels to pods.

A bit more details below. @J-Siu I hope it will explain why you are experiencing this.

---

When you run:

kubectl run ghost --image=ghost --port 2368 --expose=true

It creates a service and a pod (and a deployment, but it's not important for this issue). Important thing is - It does not assign new labels to the service implicitly. The selector you see in the output of kubectl get svc -o wide is the selector that your service uses to find related pods.

Try run kubectl get svc ghost -oyaml and you will see something like this:

apiVersion: v1
kind: Service
metadata:
  creationTimestamp: 2018-10-06T10:07:07Z
  name: ghost
  namespace: default
  resourceVersion: "110979"
  selfLink: /api/v1/namespaces/default/services/ghost
  uid: 94d84ee9-c94f-11e8-9ae1-0800274eddd9
spec:
  clusterIP: 10.106.181.96
  ports:
  - port: 2368
    protocol: TCP
    targetPort: 2368
  selector:
    run: ghost
# ^^^^^^^^^^^^
# This is what `kubectl get svc -o wide` prints. Resource itself doesn't have any labels
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}

(see the comment in the yaml file)

In order to be able to select the service by a label, you need to add a new label to it explicitly. Like kubectl label svc ghost app=something.

It works with the k8s-app=kube-dns, because the kube-dns service has this label. See:

kubectl get svc -n kube-system kube-dns -oyaml
apiVersion: v1
kind: Service
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"addonmanager.kubernetes.io/mode":"Reconcile","k8s-app":"kube-dns","kubernetes.io/name":"KubeDNS"},"name":"kube-dns","namespace":"kube-system"},"spec":{"clusterIP":"10.96.0.10","ports":[{"name":"dns","port":53,"protocol":"UDP"},{"name":"dns-tcp","port":53,"protocol":"TCP"}],"selector":{"k8s-app":"kube-dns"}}}
  creationTimestamp: 2018-08-12T19:54:21Z
  labels:
    addonmanager.kubernetes.io/mode: Reconcile
    k8s-app: kube-dns
    kubernetes.io/name: KubeDNS
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# This is what `kubectl` uses to find resources
  name: kube-dns
  namespace: kube-system
  resourceVersion: "16567"
  selfLink: /api/v1/namespaces/kube-system/services/kube-dns
  uid: 815573bc-9e69-11e8-a6b0-0800274eddd9
spec:
  clusterIP: 10.96.0.10
  ports:
  - name: dns
    port: 53
    protocol: UDP
    targetPort: 53
  - name: dns-tcp
    port: 53
    protocol: TCP
    targetPort: 53
  selector:
    k8s-app: kube-dns
# ^^^^^^^^^^^^^^^^^^^^
# This is what services uses to discover related pods
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}

(also see the comments in the yaml file)

So kube-dns just uses the same labels to mark services, pods and deployments. That's why you can see results when you run

kubectl get svc -o wide --all-namespaces -l k8s-app=kube-dns

[J-Siu]

@m1kola

Arrrrr. I understand what you saying after I went through it a few times. As I am new to kubernetes and confusing between labels/selectors and also the -l option. But after some search and try with --show-labels, it becomes clear:

$ kubectl get svc -o wide --all-namespaces --show-labels
NAMESPACE     NAME            TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE       SELECTOR               LABELS
default       ghost           ClusterIP      10.97.101.35    <none>        2368/TCP         13d       run=ghost              <none>
default       hn              LoadBalancer   10.96.66.191    <pending>     8080:30673/TCP   12d       run=hn                 run=hn
default       kubernetes      ClusterIP      10.96.0.1       <none>        443/TCP          14d       <none>                 component=apiserver,provider=kubernetes
kube-system   kube-dns        ClusterIP      10.96.0.10      <none>        53/UDP,53/TCP    14d       k8s-app=kube-dns       k8s-app=kube-dns,kubernetes.io/cluster-service=true,kubernetes.io/name=KubeDNS
kube-system   tiller-deploy   ClusterIP      10.104.224.90   <none>        44134/TCP        13d       app=helm,name=tiller   app=helm,name=tiller

Thank you!!