Metrics-server: unable to fetch pod metrics for pod/node: no metrics known for pod/node in kubernetes v1.10.3

[zuoinlv]

I deployed the metric server in kubernetes(v1.10.3). the metric server runs well. after I run the following commands, it raised the errors. if I deploy the same code in kubernetes(v1.11.1), it will work well

kube-master:~$ kubectl top pods
W1212 17:36:06.953604     726 top_pod.go:259] Metrics not available for pod default/glusterfs-fkpzt, age: 2023h57m28.953555742s
error: Metrics not available for pod default/glusterfs-fkpzt, age: 2023h57m28.953555742s

logs of metric server:

E1212 09:36:06.882322       1 reststorage.go:98] unable to fetch pod metrics for pod default/kafka-2: no metrics known for pod "default/kafka-2"
E1212 09:36:06.882385       1 reststorage.go:98] unable to fetch pod metrics for pod default/kafka-0: no metrics known for pod "default/kafka-0"
E1212 09:36:06.882405       1 reststorage.go:98] unable to fetch pod metrics for pod default/glusterfs-fkpzt: no metrics known for pod "default/glusterfs-fkpzt"
E1212 09:36:06.882421       1 reststorage.go:98] unable to fetch pod metrics for pod default/heketi-75dcfb7d44-97w27: no metrics known for pod "default/heketi-75dcfb7d44-97w27"
E1212 09:36:06.882430       1 reststorage.go:98] unable to fetch pod metrics for pod default/zk-1: no metrics known for pod "default/zk-1"
E1212 09:36:06.882438       1 reststorage.go:98] unable to fetch pod metrics for pod default/zk-0: no metrics known for pod "default/zk-0"
E1212 09:36:06.882445       1 reststorage.go:98] unable to fetch pod metrics for pod default/kafka-1: no metrics known for pod "default/kafka-1"
E1212 09:36:06.882453       1 reststorage.go:98] unable to fetch pod metrics for pod default/glusterfs-mpkvf: no metrics known for pod "default/glusterfs-mpkvf"
E1212 09:36:06.882460       1 reststorage.go:98] unable to fetch pod metrics for pod default/glusterfs-tl72x: no metrics known for pod "default/glusterfs-tl72x"
E1212 09:36:06.882468       1 reststorage.go:98] unable to fetch pod metrics for pod default/zk-2: no metrics known for pod "default/zk-2"
E1212 09:36:23.748694       1 manager.go:118] unable to save metrics: duplicate pod default/zk-2 received
E1212 09:37:23.765693       1 manager.go:118] unable to save metrics: duplicate pod default/zk-1 received
E1212 09:38:23.749128       1 manager.go:118] unable to save metrics: duplicate pod default/zk-2 received

[zuoinlv]

Seems the the log "unable to save metrics: duplicate pod default/zk-2 received" is the root cause, after I delete the pod default/zk-2 and create it again, the kube top works.

[realdimas]

Stumbled upon the same issue, also with a StatefulSet pod.
metrics-server 0.3.3 stopped returning metrics for recently launched pods and nodes.
Restarting the daemon made things worse – almost no metrics were returned.
Downgrading from 0.3.3 to 0.3.2 or 0.3.1 did not affect the issue, but deleting offending pod did resume metrics flow.

Full error messages for SEO:
metrics-server E0711 17:31:06.024250 1 manager.go:127] unable to save metrics: duplicate pod *** received
metrics-server E0711 17:36:23.384043 1 reststorage.go:128] unable to fetch node metrics for node ***: no metrics known for node
metrics-server E0711 17:36:23.923477 1 reststorage.go:147] unable to fetch pod metrics for pod ***: no metrics known for pod

Another issue mentioning duplicate pod errors along with unable to fetch pod metrics:

179 ("unable to save metrics: duplicate pod received").

Related to PR #65, commit 05fd3f6 specifically: "Rework internal metrics pipeline".