Metrics-server: unable to fully collect metrics
use k8s v1.11.2 && metrics-server v0.3.0
Did the following:
- Add apiserver args like this:
KUBE_API_ARGS="--authorization-mode=Node,RBAC --runtime-config=rbac.authorization.k8s.io/v1beta1 --kubelet-https=true --enable-bootstrap-token-auth --token-auth-file=/etc/kubernetes/token.csv --service-node-port-range=30000-32767 --tls-cert-file=/etc/kubernetes/ssl/kubernetes.pem --tls-private-key-file=/etc/kubernetes/ssl/kubernetes-key.pem --client-ca-file=/etc/kubernetes/ssl/ca.pem --service-account-key-file=/etc/kubernetes/ssl/ca-key.pem --etcd-cafile=/etc/kubernetes/ssl/ca.pem --etcd-certfile=/etc/kubernetes/ssl/kubernetes.pem --etcd-keyfile=/etc/kubernetes/ssl/kubernetes-key.pem --enable-swagger-ui=true --apiserver-count=3 --audit-log-maxage=30 --audit-log-maxbackup=3 --audit-log-maxsize=100 --audit-log-path=/var/lib/audit.log --event-ttl=1h --requestheader-client-ca-file=/etc/kubernetes/ssl/ca.pem --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --proxy-client-cert-file=/etc/kubernetes/ssl/kube-proxy.pem --proxy-client-key-file=/etc/kubernetes/ssl/kube-proxy-key.pem" - Add kubelet args like this:
KUBELET_ARGS="--network-plugin=cni --cgroup-driver=cgroupfs --cluster-dns=10.254.0.2 --bootstrap-kubeconfig=/etc/kubernetes/bootstrap.kubeconfig --kubeconfig=/etc/kubernetes/kubelet.kubeconfig --cert-dir=/etc/kubernetes/ssl --cluster-domain=cluster.local --hairpin-mode promiscuous-bridge --serialize-image-pulls=false --authentication-token-webhook=true --read-only-port=10255" - Ran kubectl create -f deploy/1.8+/
- look metrics-server logs;
then metrics-server logs show this:
unable to fully collect metrics: [unable to fully scrape metrics from source kubelet_summary:192.168.66.66: unable to fetch metrics from Kubelet 192.168.66.66 (192.168.66.66): Get https://192.168.66.66:10250/stats/summary/: x509: cannot validate certificate for 192.168.66.66 because it doesn't contain any IP SANs, unable to fully scrape metrics from source kubelet_summary:192.168.66.64: unable to fetch metrics from Kubelet 192.168.66.64 (192.168.66.64): Get https://192.168.66.64:10250/stats/summary/: x509: certificate signed by unknown authority, unable to fully scrape metrics from source kubelet_summary:192.168.220.50: unable to fetch metrics from Kubelet 192.168.220.50 (192.168.220.50): Get https://192.168.220.50:10250/stats/summary/: x509: cannot validate certificate for 192.168.220.50 because it doesn't contain any IP SANs]
when I use metrics-server v0.2.1,add metrcs-server args like this:
command:
- /metrics-server
- --source=kubernetes.summary_api:https://kubernetes.default.svc?kubeletHttps=true&kubeletPort=10250&useServiceAccount=true&insecure=true
It seems to work,but when i use v0.3.0 and add args --source it's says Error: unknown flag: --source,
So I used the help instruction to find the corresponding parameter, but I didn't find it. So, how can I fix this problem?
This is my first time to mention issue, which is not so good. Sorry.
kinsupport
vvkkhjt
👍4
All 32 comments
@vvkkhjt try this flag --kubelet-insecure-tls
MIBc
on 3 Sep 2018
👍4
the --kubelet-insecure-tls flag is the equivalent of the insecure=true flag in the old metrics server release. However, if you're going to connect via internal IP addresses, I'd recommended just setting up your certificates so that they list the internal IP as an alternate name.
DirectXMan12
on 5 Sep 2018
@MIBc when i use that flag:
streamwatcher.go:109] Unable to decode an event from the watch stream: http2: server sent GOAWAY and closed the connection; LastStreamID=15, ErrCode=NO_ERROR, debug=""
E0906 06:28:39.309417 1 streamwatcher.go:109] Unable to decode an event from the watch stream: http2: server sent GOAWAY and closed the connection; LastStreamID=15, ErrCode=NO_ERROR, debug=""
E0906 06:28:39.312965 1 reflector.go:322] github.com/kubernetes-incubator/metrics-server/vendor/k8s.io/client-go/informers/factory.go:130: Failed to watch *v1.Pod: Get https://10.254.0.1:443/api/v1/pods?resourceVersion=856148&timeoutSeconds=395&watch=true: dial tcp 10.254.0.1:443: connect: connection refused
E0906 06:28:39.313303 1 reflector.go:322] github.com/kubernetes-incubator/metrics-server/vendor/k8s.io/client-go/informers/factory.go:130: Failed to watch *v1.Node: Get https://10.254.0.1:443/api/v1/nodes?resourceVersion=858077&timeoutSeconds=440&watch=true: dial tcp 10.254.0.1:443: connect: connection refused
Get https://10.254.0.1:443/api/v1/pods?resourceVersion=856148&timeoutSeconds=395&watch=true: dial tcp 10.254.0.1:443: connect: connection refused
That implies that metrics-server isn't able to contact your main kubernetes API server.
DirectXMan12
on 6 Sep 2018
@DirectXMan12 but the other services are working good。。when i use helm install heapster,it works good either。
helm install stable/heapster --name heapster --namespace kube-system --set "rbac.create=true,command[0]=/heapster,command[1]=--source=kubernetes.summary_api:https://kubernetes.default.svc?kubeletHttps=true&kubeletPort=10250&useServiceAccount=true&insecure=true" --debug
then
k top node
NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
192.168.220.50 254m 3% 3606Mi 11%
192.168.66.64 226m 5% 2096Mi 57%
192.168.66.66 99m 2% 1388Mi 62%
👍1
Hi all,
k8s: v1.11
metrics-server: 0.3.1
I am not using TLS, However, I fall into similar problem(not being able to fully collect metrics). The logs of metrics-server show that it receives a - Raw (json) response - from kubelet, but i can't retrieve the metrics via k top node/pod.
After receiving the kubelet's sumary(json) there is such a line:
ScrapeMetrics: time: 59.513576ms, nodes: 0, pods: 0
Another message says it cannot parse some init containers. Is this the reason it is failing?
unable to get a valid timestamp for metric point for container
Best!
E0924 12:45:08.398855 1 manager.go:102] unable to fully collect metrics: [unable to fully scrape metrics from source kubelet_summary:k8s1: unable to get a valid timestamp for metric point for container "install-cni" in pod kube-system/kube-router-2l6f5 on node "10.36.10.1", discarding data: no non-zero timestamp on either CPU or memory, unable to fully scrape metrics from source kubelet_summary:k8s4: unable to get a valid timestamp for metric point for container "install-cni" in pod kube-system/kube-router-6zvxd on node "10.36.10.4", discarding data: no non-zero timestamp on either CPU or memory, unable to fully scrape
metrics from source kubelet_summary:k8s5: [unable to get a valid timestamp for metric point for container "compile" in pod default/hello-6786bd6b44-t6z5z on node "10.36.10.5", discarding data: no non-zero timestamp on either CPU or memory, unable to get a valid timestamp for metric point for container "prepare" in pod default/hello-6786bd6b44-t6z5z on node "10.36.10.5", discarding data: no non-zero timestamp on either
CPU or memory, unable to get a valid timestamp for metric point for container "install-cni" in pod kube-system/kube-router-gntmq on node "10.36.10.5", discarding data: no non-zero timestamp on either CPU or memory]]
brunovlucena
on 24 Sep 2018
👍3
@brunovlucena what happens when you run k top node? It's expected that right after pod start there'll be missing points, but you should still be able to get information for other pods/nodes.
DirectXMan12
on 24 Sep 2018
i have the same problem
gabrielfsousa
on 25 Sep 2018
@brunovlucena what happens when you run
k top node? It's expected that right after pod start there'll be missing points, but you should still be able to get information for other pods/nodes.
Error from server (NotFound): nodemetrics.metrics.k8s.io "k8s1" not found
brunovlucena
on 26 Sep 2018
is there only one node? kubectl top is pretty finnicky about partial data, so it'd be good to see if there's partial data being returned (try kubectl top node --v=10).
DirectXMan12
on 27 Sep 2018
kubectl top node k8s1 --heapster-service=metrics-server --heapster-port= --heapster-namespace=kube-system --heapster-scheme= --v=10
I1002 11:53:59.648863 8979 request.go:897] Response Body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"nodemetrics.metrics.k8s.io \"k8s1\" not found","reason":"NotFound","details":{"name":"k8s1","group":"metrics.k8s.io","kind":"nodemetrics"},"code":404}
I1002 11:53:59.649584 8979 helpers.go:201] server response object: [{
"metadata": {},
"status": "Failure",
"message": "nodemetrics.metrics.k8s.io \"k8s1\" not found",
"reason": "NotFound",
"details": {
"name": "k8s1",
"group": "metrics.k8s.io",
"kind": "nodemetrics"
},
"code": 404
}]
kubectl get --raw "/apis/metrics.k8s.io/v1beta1/nodes" | jq
{
"kind": "NodeMetricsList",
"apiVersion": "metrics.k8s.io/v1beta1",
"metadata": {
"selfLink": "/apis/metrics.k8s.io/v1beta1/nodes"
},
"items": []
}
kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s1 Ready <none> 20m v1.11.2
k8s4 Ready <none> 20m v1.11.2
k8s5 Ready <none> 20m v1.11.2
kubectl get ep --all-namespaces -o wide
NAMESPACE NAME ENDPOINTS AGE
default kubernetes 10.36.10.1:6443,10.36.10.4:6443,10.36.10.5:6443 21m
kube-system kube-controller-manager <none> 21m
kube-system kube-dns 10.200.0.2:53,10.200.1.3:53,10.200.0.2:53 + 1 more... 9s
kube-system kube-scheduler <none> 21m
kube-system metrics-server 10.200.1.2:443 6m
That kubectl top node output looks truncated, and seems to have extra heapster arguments. Is that the full output? If not, can you post the full output, and if so, can you try without the extra heapster arguments?
DirectXMan12
on 2 Oct 2018
@DirectXMan12 I have tried the --kubelet-insecure-tls flag, but heapster fails to start and in the logs I can see a "usage" screen telling me "unknown flag: --kubelet-insecure-tls".
I have also tried to add that on the line where the source is defined, like this:
- --source=kubernetes.summary_api:https://kubernetes.default?kubeletHttps=true&kubeletPort=10250&kubelet-insecure-tls=true
In this case heapster will start, but will still fail to retrieve metrics due to lack of IP SAN in the kubelet certs.
I've tried with both v1.4.0 and v1.5.4 of heapster.
Is there a way to tell heapster to accept the cert without that IP SAN setting?
Thank you,
Laszlo
lbudai74
on 5 Oct 2018
i have the same problem...
tengzhexiao
on 12 Oct 2018
@lbudai74 this is the repository for metrics-server, not heapster. Heapster's repository can be found at kubernetes/heapster.
DirectXMan12
on 8 Nov 2018
@tengzhexiao which problem?
DirectXMan12
on 8 Nov 2018
I am also facing the same issue
metrics-server logs ::
I1129 06:48:29.089117 1 serving.go:273] Generated self-signed cert (apiserver.local.config/certificates/apiserver.crt, apiserver.local.config/certificates/apiserver.key)
[restful] 2018/11/29 06:48:30 log.go:33: [restful/swagger] listing is available at https://:443/swaggerapi
[restful] 2018/11/29 06:48:30 log.go:33: [restful/swagger] https://:443/swaggerui/ is mapped to folder /swagger-ui/
I1129 06:48:30.183062 1 serve.go:96] Serving securely on [::]:443
E1129 06:49:30.145670 1 manager.go:102] unable to fully collect metrics: unable to fully scrape metrics from source kubelet_summary:127.0.0.1: unable to fetch metrics from Kubelet 127.0.0.1 (127.0.0.1): Get https://127.0.0.1:10250/stats/summary/: dial tcp 127.0.0.1:10250: connect: connection refused
E1129 06:50:30.139404 1 manager.go:102] unable to fully collect metrics: unable to fully scrape metrics from source kubelet_summary:127.0.0.1: unable to fetch metrics from Kubelet 127.0.0.1 (127.0.0.1): Get https://127.0.0.1:10250/stats/summary/: dial tcp 127.0.0.1:10250: connect: connection refused
Rajat-0
on 29 Nov 2018
@Rajat-0 I think you're facing a different issue. If your kubelet lists the preferred address as 127.0.0.1, metrics-server can't collect from it, because it's the 127.0.0.1 is the loopback adapter (i.e. localhost in the pod is not localhost in the node).
DirectXMan12
on 30 Nov 2018
i have the same problem
lolspider
on 3 Dec 2018
what about a feature that has watches a namespace for a configmap per node that contains the ca to use to verify that node? Nodes can self register self signed ca's then.
kfox1111
on 6 Feb 2019
well I have the same issue, opened a ticket here: #214 tried to be as much as informative as possible.
so this is what happens with v0.2.1 - when i run watch 'kubectl top pod
in v0.3.1 it ignores completely from pods with init containers. and why is that ? because init containers returns empty stats in stats summary api call (unless you are using docker on the kubernetes nodes).
and I think it's because when parsing the json in the go code of metrics-server it drops the pods with init containers entirely .
shahaf600
on 7 Feb 2019
@DirectXMan12
Provider: AWS EKS
K8s version: 1.11.5
metrics-server: 0.3.1
Seems having similar issue with @brunovlucena but doesn't look that is related with heapster, even If I run also heapster apart from metrics-server because without heapster my kubernetes dashboard graphs were not working.
The verbose outpout you ask from @brunovlucena in my case is the following.
I0211 10:34:04.021131 11131 round_trippers.go:411] Response Headers:
I0211 10:34:04.021143 11131 round_trippers.go:414] Audit-Id: 52cf9a16-77ce-48ca-a6c9-474626e3ca79
I0211 10:34:04.021168 11131 round_trippers.go:414] Content-Type: text/plain; charset=utf-8
I0211 10:34:04.021179 11131 round_trippers.go:414] X-Content-Type-Options: nosniff
I0211 10:34:04.021189 11131 round_trippers.go:414] Content-Length: 20
I0211 10:34:04.021198 11131 round_trippers.go:414] Date: Mon, 11 Feb 2019 08:34:03 GMT
I0211 10:34:04.021234 11131 request.go:942] Response Body: service unavailable
I0211 10:34:04.021434 11131 helpers.go:198] server response object: [{
"metadata": {},
"status": "Failure",
"message": "the server is currently unable to handle the request (get pods.metrics.k8s.io)",
"reason": "ServiceUnavailable",
"details": {
"group": "metrics.k8s.io",
"kind": "pods",
"causes": [
{
"reason": "UnexpectedServerResponse",
"message": "service unavailable"
}
]
},
"code": 503
}]
F0211 10:34:04.021487 11131 helpers.go:116] Error from server (ServiceUnavailable): the server is currently unable to handle the request (get pods.metrics.k8s.io)
Any ideas would be appreciated.
stafot
on 11 Feb 2019
@DirectXMan12
Provider: AWS EKS
K8s version: 1.11.5
metrics-server: 0.3.1
Seems having similar issue with @brunovlucena but doesn't look that is related with heapster, even If I run also heapster apart from metrics-server because without heapster my kubernetes dashboard graphs were not working.
The verbose outpout you ask from @brunovlucena in my case is the following.I0211 10:34:04.021131 11131 round_trippers.go:411] Response Headers: I0211 10:34:04.021143 11131 round_trippers.go:414] Audit-Id: 52cf9a16-77ce-48ca-a6c9-474626e3ca79 I0211 10:34:04.021168 11131 round_trippers.go:414] Content-Type: text/plain; charset=utf-8 I0211 10:34:04.021179 11131 round_trippers.go:414] X-Content-Type-Options: nosniff I0211 10:34:04.021189 11131 round_trippers.go:414] Content-Length: 20 I0211 10:34:04.021198 11131 round_trippers.go:414] Date: Mon, 11 Feb 2019 08:34:03 GMT I0211 10:34:04.021234 11131 request.go:942] Response Body: service unavailable I0211 10:34:04.021434 11131 helpers.go:198] server response object: [{ "metadata": {}, "status": "Failure", "message": "the server is currently unable to handle the request (get pods.metrics.k8s.io)", "reason": "ServiceUnavailable", "details": { "group": "metrics.k8s.io", "kind": "pods", "causes": [ { "reason": "UnexpectedServerResponse", "message": "service unavailable" } ] }, "code": 503 }] F0211 10:34:04.021487 11131 helpers.go:116] Error from server (ServiceUnavailable): the server is currently unable to handle the request (get pods.metrics.k8s.io)Any ideas would be appreciated.
when i use kubectl top node ,I get the same error like you ,have you fix it ?
Kingchan230
on 28 Feb 2019
@Kingchan230 Yes I resolved my issue. In my case the problem was that I applied some resources from kube-prometheus that messed my infra. At this implementation prometheus-adapter instead of using v1beta1.custom.metrics.k8s.io uses v1beta1.metrics.k8s.io and I did not applied a prometheus-adapter service. I don't know if you are in the same situation. But to help you debug it, in my case the command that lead me to understand my misconfiguration was the below one. kubectl describe apiservice v1beta1.metrics.k8s.io which returned the following, inter alla
Message: service/prometheus-adapter in "monitoring" is not present
Reason: ServiceNotFound
Status: False
Type: Available
If something is unclear feel free to reach me back.
stafot
on 28 Feb 2019
i have the same problem
E0404 07:03:00.095957 1 manager.go:102] unable to fully collect metrics: unable to fully scrape metrics from source kubelet_summary:kubernetes-113: unable to fetch metrics from Kubelet kubernetes-113 (10.180.160.113): request failed - "500 Internal Server Error", response: "Authorization error (user=system:serviceaccount:kube-system:metrics-server, verb=get, resource=nodes, subresource=stats)"
tonglv
on 8 Apr 2019
I had the same issue and re-applying the Prometheus adapter API service file fixed it:
kubectl apply -f prometheus-adapter-apiService.yaml
ksylvan
on 18 Jun 2019
FIxed for me on EKS
helm install stable/metrics-server --name metrics-server --namespace metrics --set args[0]=--kubelet-insecure-tls,args[1]=--kubelet-preferred-address-types=InternalIP
urvineet
on 5 Jul 2019
👍3
Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale
fejta-bot
on 3 Oct 2019
/remove-lifecycle stale
vvkkhjt
on 10 Oct 2019
@DirectXMan12 I use kubectl top node --v=10 command output information is as follows:
[root@node40 ~]# kubectl top node --v=10
I1028 17:02:10.710211 2914 loader.go:359] Config loaded from file /root/.kube/config
I1028 17:02:10.714789 2914 loader.go:359] Config loaded from file /root/.kube/config
I1028 17:02:10.715216 2914 round_trippers.go:419] curl -k -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.13.6 (linux/amd64) kubernetes/abdda3f" 'https://192.168.0.50:8443/api?timeout=32s'
I1028 17:02:10.728855 2914 round_trippers.go:438] GET https://192.168.0.50:8443/api?timeout=32s 200 OK in 13 milliseconds
I1028 17:02:10.728981 2914 round_trippers.go:444] Response Headers:
I1028 17:02:10.729031 2914 round_trippers.go:447] Content-Length: 135
I1028 17:02:10.729076 2914 round_trippers.go:447] Date: Mon, 28 Oct 2019 09:02:10 GMT
I1028 17:02:10.729140 2914 round_trippers.go:447] Content-Type: application/json
I1028 17:02:10.729241 2914 request.go:942] Response Body: {"kind":"APIVersions","versions":["v1"],"serverAddressByClientCIDRs":[{"clientCIDR":"0.0.0.0/0","serverAddress":"192.168.0.41:6443"}]}
I1028 17:02:10.729562 2914 round_trippers.go:419] curl -k -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.13.6 (linux/amd64) kubernetes/abdda3f" 'https://192.168.0.50:8443/apis?timeout=32s'
I1028 17:02:10.730976 2914 round_trippers.go:438] GET https://192.168.0.50:8443/apis?timeout=32s 200 OK in 1 milliseconds
I1028 17:02:10.731137 2914 round_trippers.go:444] Response Headers:
I1028 17:02:10.731203 2914 round_trippers.go:447] Date: Mon, 28 Oct 2019 09:02:10 GMT
I1028 17:02:10.731299 2914 round_trippers.go:447] Content-Type: application/json
I1028 17:02:10.731361 2914 round_trippers.go:447] Content-Length: 4054
I1028 17:02:10.731480 2914 request.go:942] Response Body: {"kind":"APIGroupList","apiVersion":"v1","groups":[{"name":"apiregistration.k8s.io","versions":[{"groupVersion":"apiregistration.k8s.io/v1","version":"v1"},{"groupVersion":"apiregistration.k8s.io/v1beta1","version":"v1beta1"}],"preferredVersion":{"groupVersion":"apiregistration.k8s.io/v1","version":"v1"}},{"name":"extensions","versions":[{"groupVersion":"extensions/v1beta1","version":"v1beta1"}],"preferredVersion":{"groupVersion":"extensions/v1beta1","version":"v1beta1"}},{"name":"apps","versions":[{"groupVersion":"apps/v1","version":"v1"},{"groupVersion":"apps/v1beta2","version":"v1beta2"},{"groupVersion":"apps/v1beta1","version":"v1beta1"}],"preferredVersion":{"groupVersion":"apps/v1","version":"v1"}},{"name":"events.k8s.io","versions":[{"groupVersion":"events.k8s.io/v1beta1","version":"v1beta1"}],"preferredVersion":{"groupVersion":"events.k8s.io/v1beta1","version":"v1beta1"}},{"name":"authentication.k8s.io","versions":[{"groupVersion":"authentication.k8s.io/v1","version":"v1"},{"groupVersion":"authentication.k8s.io/v1beta1","version":"v1beta1"}],"preferredVersion":{"groupVersion":"authentication.k8s.io/v1","version":"v1"}},{"name":"authorization.k8s.io","versions":[{"groupVersion":"authorization.k8s.io/v1","version":"v1"},{"groupVersion":"authorization.k8s.io/v1beta1","version":"v1beta1"}],"preferredVersion":{"groupVersion":"authorization.k8s.io/v1","version":"v1"}},{"name":"autoscaling","versions":[{"groupVersion":"autoscaling/v1","version":"v1"},{"groupVersion":"autoscaling/v2beta1","version":"v2beta1"},{"groupVersion":"autoscaling/v2beta2","version":"v2beta2"}],"preferredVersion":{"groupVersion":"autoscaling/v1","version":"v1"}},{"name":"batch","versions":[{"groupVersion":"batch/v1","version":"v1"},{"groupVersion":"batch/v1beta1","version":"v1beta1"}],"preferredVersion":{"groupVersion":"batch/v1","version":"v1"}},{"name":"certificates.k8s.io","versions":[{"groupVersion":"certificates.k8s.io/v1beta1","version":"v1beta1"}],"preferredVersion":{"groupVersion":"certificates.k8s.io/v1beta1","version":"v1beta1"}},{"name":"networking.k8s.io","versions":[{"groupVersion":"networking.k8s.io/v1","version":"v1"}],"preferredVersion":{"groupVersion":"networking.k8s.io/v1","version":"v1"}},{"name":"policy","versions":[{"groupVersion":"policy/v1beta1","version":"v1beta1"}],"preferredVersion":{"groupVersion":"policy/v1beta1","version":"v1beta1"}},{"name":"rbac.authorization.k8s.io","versions":[{"groupVersion":"rbac.authorization.k8s.io/v1","version":"v1"},{"groupVersion":"rbac.authorization.k8s.io/v1beta1","version":"v1beta1"}],"preferredVersion":{"groupVersion":"rbac.authorization.k8s.io/v1","version":"v1"}},{"name":"settings.k8s.io","versions":[{"groupVersion":"settings.k8s.io/v1alpha1","version":"v1alpha1"}],"preferredVersion":{"groupVersion":"settings.k8s.io/v1alpha1","version":"v1alpha1"}},{"name":"storage.k8s.io","versions":[{"groupVersion":"storage.k8s.io/v1","version":"v1"},{"groupVersion":"storage.k8s.io/v1beta1","version":"v1beta1"}],"preferredVersion":{"groupVersion":"storage.k8s.io/v1","version":"v1"}},{"name":"admissionregistration.k8s.io","versions":[{"groupVersion":"admissionregistration.k8s.io/v1beta1","version":"v1beta1"}],"preferredVersion":{"groupVersion":"admissionregistration.k8s.io/v1beta1","version":"v1beta1"}},{"name":"apiextensions.k8s.io","versions":[{"groupVersion":"apiextensions.k8s.io/v1beta1","version":"v1beta1"}],"preferredVersion":{"groupVersion":"apiextensions.k8s.io/v1beta1","version":"v1beta1"}},{"name":"scheduling.k8s.io","versions":[{"groupVersion":"scheduling.k8s.io/v1beta1","version":"v1beta1"}],"preferredVersion":{"groupVersion":"scheduling.k8s.io/v1beta1","version":"v1beta1"}},{"name":"coordination.k8s.io","versions":[{"groupVersion":"coordination.k8s.io/v1beta1","version":"v1beta1"}],"preferredVersion":{"groupVersion":"coordination.k8s.io/v1beta1","version":"v1beta1"}},{"name":"metrics.k8s.io","versions":[{"groupVersion":"metrics.k8s.io/v1beta1","version":"v1beta1"}],"preferredVersion":{"groupVersion":"metrics.k8s.io/v1beta1","version":"v1beta1"}}]}
I1028 17:02:10.732013 2914 round_trippers.go:419] curl -k -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.13.6 (linux/amd64) kubernetes/abdda3f" 'https://192.168.0.50:8443/apis/metrics.k8s.io/v1beta1/nodes'
I1028 17:02:10.735511 2914 round_trippers.go:438] GET https://192.168.0.50:8443/apis/metrics.k8s.io/v1beta1/nodes 200 OK in 3 milliseconds
I1028 17:02:10.735538 2914 round_trippers.go:444] Response Headers:
I1028 17:02:10.735547 2914 round_trippers.go:447] Date: Mon, 28 Oct 2019 09:02:10 GMT
I1028 17:02:10.735553 2914 round_trippers.go:447] Content-Length: 375
I1028 17:02:10.735559 2914 round_trippers.go:447] Content-Type: application/json
I1028 17:02:10.735588 2914 request.go:942] Response Body: {"kind":"NodeMetricsList","apiVersion":"metrics.k8s.io/v1beta1","metadata":{"selfLink":"/apis/metrics.k8s.io/v1beta1/nodes"},"items":[{"metadata":{"name":"192.168.0.47","selfLink":"/apis/metrics.k8s.io/v1beta1/nodes/192.168.0.47","creationTimestamp":"2019-10-28T09:02:10Z"},"timestamp":"2019-10-28T09:01:13Z","window":"30s","usage":{"cpu":"63913024n","memory":"577908Ki"}}]}
I1028 17:02:10.736558 2914 round_trippers.go:419] curl -k -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.13.6 (linux/amd64) kubernetes/abdda3f" 'https://192.168.0.50:8443/api/v1/nodes'
I1028 17:02:10.741531 2914 round_trippers.go:438] GET https://192.168.0.50:8443/api/v1/nodes 200 OK in 4 milliseconds
I1028 17:02:10.741556 2914 round_trippers.go:444] Response Headers:
I1028 17:02:10.741565 2914 round_trippers.go:447] Content-Type: application/json
I1028 17:02:10.741572 2914 round_trippers.go:447] Date: Mon, 28 Oct 2019 09:02:10 GMT
I1028 17:02:10.743772 2914 request.go:942] Response Body: {"kind":"NodeList","apiVersion":"v1","metadata":{"selfLink":"/api/v1/nodes","resourceVersion":"1940342"},"items":[{"metadata":{"name":"192.168.0.40","selfLink":"/api/v1/nodes/192.168.0.40","uid":"5e1f1f10-f182-11e9-84fc-902b34d396c8","resourceVersion":"1940340","creationTimestamp":"2019-10-18T08:36:23Z","labels":{"beta.kubernetes.io/arch":"amd64","beta.kubernetes.io/os":"linux","kubernetes.io/hostname":"192.168.0.40","node-role.kubernetes.io/master":"node40"},"annotations":{"node.alpha.kubernetes.io/ttl":"0","volumes.kubernetes.io/controller-managed-attach-detach":"true"}},"spec":{"podCIDR":"10.20.0.0/24","taints":[{"key":"node-role.kubernetes.io/master","value":"node40","effect":"NoSchedule"}]},"status":{"capacity":{"cpu":"4","ephemeral-storage":"79833Mi","hugepages-2Mi":"0","memory":"16296408Ki","pods":"200"},"allocatable":{"cpu":"4","ephemeral-storage":"75339870903","hugepages-2Mi":"0","memory":"16194008Ki","pods":"200"},"conditions":[{"type":"MemoryPressure","status":"False","lastHeartbeatTime":"2019-10-28T09:02:10Z","lastTransitionTime":"2019-10-28T08:48:09Z","reason":"KubeletHasSufficientMemory","message":"kubelet has sufficient memory available"},{"type":"DiskPressure","status":"False","lastHeartbeatTime":"2019-10-28T09:02:10Z","lastTransitionTime":"2019-10-28T08:48:09Z","reason":"KubeletHasNoDiskPressure","message":"kubelet has no disk pressure"},{"type":"PIDPressure","status":"False","lastHeartbeatTime":"2019-10-28T09:02:10Z","lastTransitionTime":"2019-10-28T08:48:09Z","reason":"KubeletHasSufficientPID","message":"kubelet has sufficient PID available"},{"type":"Ready","status":"True","lastHeartbeatTime":"2019-10-28T09:02:10Z","lastTransitionTime":"2019-10-28T08:48:09Z","reason":"KubeletReady","message":"kubelet is posting ready status"}],"addresses":[{"type":"InternalIP","address":"192.168.0.40"},{"type":"Hostname","address":"192.168.0.40"}],"daemonEndpoints":{"kubeletEndpoint":{"Port":10250}},"nodeInfo":{"machineID":"8f57bb01b7084c389d564b784f8061c9","systemUUID":"032b0290-0434-05d3-9606-c80700080009","bootID":"9e985074-6ec7-4b83-bb10-abfe3949c0fe","kernelVersion":"4.19.12-1.el7.elrepo.x86_64","osImage":"CentOS Linux 7 (Core)","containerRuntimeVersion":"docker://18.6.3","kubeletVersion":"v1.13.6","kubeProxyVersion":"v1.13.6","operatingSystem":"linux","architecture":"amd64"},"images":[{"names":["quay.io/external_storage/rbd-provisioner@sha256:94fd36b8625141b62ff1addfa914d45f7b39619e55891bad0294263ecd2ce09a","quay.io/external_storage/rbd-provisioner:latest"],"sizeBytes":404591002},{"names":["registry.cn-hangzhou.aliyuncs.com/dnsjia/k8s@sha256:1d2e1229a918f4bc38b5a3f9f5f11302b3e71f8397b492afac7f273a0008776a","registry.cn-hangzhou.aliyuncs.com/dnsjia/k8s:kubernetes-dashboard-amd64_v1.10.0"],"sizeBytes":122460923}]}},{"metadata":{"name":"192.168.0.41","selfLink":"/api/v1/nodes/192.168.0.41","uid":"5e3a5adf-f182-11e9-84fc-902b34d396c8","resourceVersion":"1940341","creationTimestamp":"2019-10-18T08:36:24Z","labels":{"beta.kubernetes.io/arch":"amd64","beta.kubernetes.io/os":"linux","kubernetes.io/hostname":"192.168.0.41","node-role.kubernetes.io/master":"node41"},"annotations":{"node.alpha.kubernetes.io/ttl":"0","volumes.kubernetes.io/controller-managed-attach-detach":"true"}},"spec":{"podCIDR":"10.20.2.0/24","taints":[{"key":"node-role.kubernetes.io/master","value":"node41","effect":"NoSchedule"}]},"status":{"capacity":{"cpu":"4","ephemeral-storage":"79833Mi","hugepages-2Mi":"0","memory":"16296404Ki","pods":"200"},"allocatable":{"cpu":"4","ephemeral-storage":"75339870903","hugepages-2Mi":"0","memory":"16194004Ki","pods":"200"},"conditions":[{"type":"MemoryPressure","status":"False","lastHeartbeatTime":"2019-10-28T09:02:10Z","lastTransitionTime":"2019-10-28T08:29:26Z","reason":"KubeletHasSufficientMemory","message":"kubelet has sufficient memory available"},{"type":"DiskPressure","status":"False","lastHeartbeatTime":"2019-10-28T09:02:10Z","lastTransitionTime":"2019-10-28T08:29:26Z","reason":"KubeletHasNoDiskPressure","message":"kubelet has no disk pressure"},{"type":"PIDPressure","status":"False","lastHeartbeatTime":"2019-10-28T09:02:10Z","lastTransitionTime":"2019-10-28T08:29:26Z","reason":"KubeletHasSufficientPID","message":"kubelet has sufficient PID available"},{"type":"Ready","status":"True","lastHeartbeatTime":"2019-10-28T09:02:10Z","lastTransitionTime":"2019-10-28T08:29:26Z","reason":"KubeletReady","message":"kubelet is posting ready status"}],"addresses":[{"type":"InternalIP","address":"192.168.0.41"},{"type":"Hostname","address":"192.168.0.41"}],"daemonEndpoints":{"kubeletEndpoint":{"Port":10250}},"nodeInfo":{"machineID":"7069d7465cb74ff59781bec038157837","systemUUID":"032b0290-0434-05d7-9c06-490700080009","bootID":"c03c0982-3180-4acf-bdeb-03584787bf41","kernelVersion":"4.19.12-1.el7.elrepo.x86_64","osImage":"CentOS Linux 7 (Core)","containerRuntimeVersion":"docker://18.6.3","kubeletVersion":"v1.13.6","kubeProxyVersion":"v1.13.6","operatingSystem":"linux","architecture":"amd64"}}},{"metadata":{"name":"192.168.0.43","selfLink":"/api/v1/nodes/192.168.0.43","uid":"5e5422a9-f182-11e9-84fc-902b34d396c8","resourceVersion":"1940342","creationTimestamp":"2019-10-18T08:36:24Z","labels":{"beta.kubernetes.io/arch":"amd64","beta.kubernetes.io/os":"linux","kubernetes.io/hostname":"192.168.0.43","node-role.kubernetes.io/master":"node43"},"annotations":{"node.alpha.kubernetes.io/ttl":"0","volumes.kubernetes.io/controller-managed-attach-detach":"true"}},"spec":{"podCIDR":"10.20.3.0/24","taints":[{"key":"node-role.kubernetes.io/master","value":"node43","effect":"NoSchedule"}]},"status":{"capacity":{"cpu":"4","ephemeral-storage":"79833Mi","hugepages-2Mi":"0","memory":"32602520Ki","pods":"200"},"allocatable":{"cpu":"4","ephemeral-storage":"75339870903","hugepages-2Mi":"0","memory":"32500120Ki","pods":"200"},"conditions":[{"type":"MemoryPressure","status":"False","lastHeartbeatTime":"2019-10-28T09:02:10Z","lastTransitionTime":"2019-10-28T08:44:03Z","reason":"KubeletHasSufficientMemory","message":"kubelet has sufficient memory available"},{"type":"DiskPressure","status":"False","lastHeartbeatTime":"2019-10-28T09:02:10Z","lastTransitionTime":"2019-10-28T08:44:03Z","reason":"KubeletHasNoDiskPressure","message":"kubelet has no disk pressure"},{"type":"PIDPressure","status":"False","lastHeartbeatTime":"2019-10-28T09:02:10Z","lastTransitionTime":"2019-10-28T08:44:03Z","reason":"KubeletHasSufficientPID","message":"kubelet has sufficient PID available"},{"type":"Ready","status":"True","lastHeartbeatTime":"2019-10-28T09:02:10Z","lastTransitionTime":"2019-10-28T08:44:03Z","reason":"KubeletReady","message":"kubelet is posting ready status"}],"addresses":[{"type":"InternalIP","address":"192.168.0.43"},{"type":"Hostname","address":"192.168.0.43"}],"daemonEndpoints":{"kubeletEndpoint":{"Port":10250}},"nodeInfo":{"machineID":"fca78b295dc246ca8644f6fed6b145b0","systemUUID":"29c7c1c0-d7da-11dd-8c43-74d02bc70855","bootID":"9e5c2b9e-9cd5-4fc1-b197-3e83bcecbe4f","kernelVersion":"4.19.12-1.el7.elrepo.x86_64","osImage":"CentOS Linux 7 (Core)","containerRuntimeVersion":"docker://18.6.3","kubeletVersion":"v1.13.6","kubeProxyVersion":"v1.13.6","operatingSystem":"linux","architecture":"amd64"}}},{"metadata":{"name":"192.168.0.44","selfLink":"/api/v1/nodes/192.168.0.44","uid":"5e55144b-f182-11e9-84fc-902b34d396c8","resourceVersion":"1940332","creationTimestamp":"2019-10-18T08:36:24Z","labels":{"beta.kubernetes.io/arch":"amd64","beta.kubernetes.io/os":"linux","kubernetes.io/hostname":"192.168.0.44","node-role.kubernetes.io/node":"node44"},"annotations":{"node.alpha.kubernetes.io/ttl":"0","volumes.kubernetes.io/controller-managed-attach-detach":"true"}},"spec":{"podCIDR":"10.20.4.0/24"},"status":{"capacity":{"cpu":"4","ephemeral-storage":"79833Mi","hugepages-2Mi":"0","memory":"32809284Ki","pods":"200"},"allocatable":{"cpu":"4","ephemeral-storage":"75339870903","hugepages-2Mi":"0","memory":"32706884Ki","pods":"200"},"conditions":[{"type":"MemoryPressure","status":"False","lastHeartbeatTime":"2019-10-28T09:02:05Z","lastTransitionTime":"2019-10-28T08:34:28Z","reason":"KubeletHasSufficientMemory","message":"kubelet has sufficient memory available"},{"type":"DiskPressure","status":"False","lastHeartbeatTime":"2019-10-28T09:02:05Z","lastTransitionTime":"2019-10-28T08:34:28Z","reason":"KubeletHasNoDiskPressure","message":"kubelet has no disk pressure"},{"type":"PIDPressure","status":"False","lastHeartbeatTime":"2019-10-28T09:02:05Z","lastTransitionTime":"2019-10-28T08:34:28Z","reason":"KubeletHasSufficientPID","message":"kubelet has sufficient PID available"},{"type":"Ready","status":"True","lastHeartbeatTime":"2019-10-28T09:02:05Z","lastTransitionTime":"2019-10-28T08:34:28Z","reason":"KubeletReady","message":"kubelet is posting ready status"}],"addresses":[{"type":"InternalIP","address":"192.168.0.44"},{"type":"Hostname","address":"192.168.0.44"}],"daemonEndpoints":{"kubeletEndpoint":{"Port":10250}},"nodeInfo":{"machineID":"3f08673aa85c4fbbbac93b9e7ff56d74","systemUUID":"03de0294-0480-0521-1506-470700080009","bootID":"d3a83738-9f77-453f-94bd-5cc42d3a001e","kernelVersion":"4.19.12-1.el7.elrepo.x86_64","osImage":"CentOS Linux 7 (Core)","containerRuntimeVersion":"docker://18.6.3","kubeletVersion":"v1.13.6","kubeProxyVersion":"v1.13.6","operatingSystem":"linux","architecture":"amd64"},"images":[{"names":["quay.io/external_storage/rbd-provisioner@sha256:94fd36b8625141b62ff1addfa914d45f7b39619e55891bad0294263ecd2ce09a","quay.io/external_storage/rbd-provisioner:latest"],"sizeBytes":404591002},{"names":["coredns/coredns@sha256:12eb885b8685b1b13a04ecf5c23bc809c2e57917252fd7b0be9e9c00644e8ee5","coredns/coredns:1.6.2"],"sizeBytes":44100963},{"names":["radial/busyboxplus@sha256:a68c05ab1112fd90ad7b14985a48520e9d26dbbe00cb9c09aa79fdc0ef46b372","radial/busyboxplus:curl"],"sizeBytes":4233788},{"names":["registry.cn-hangzhou.aliyuncs.com/google_containers/pause-amd64@sha256:759c3f0f6493093a9043cc813092290af69029699ade0e3dbe024e968fcb7cca","registry.cn-hangzhou.aliyuncs.com/google_containers/pause-amd64:3.1"],"sizeBytes":742472}]}},{"metadata":{"name":"192.168.0.45","selfLink":"/api/v1/nodes/192.168.0.45","uid":"94b73d60-f182-11e9-a4e9-902b34d79c49","resourceVersion":"1940324","creationTimestamp":"2019-10-18T08:37:55Z","labels":{"beta.kubernetes.io/arch":"amd64","beta.kubernetes.io/os":"linux","kubernetes.io/hostname":"192.168.0.45","node-role.kubernetes.io/node":"node45"},"annotations":{"node.alpha.kubernetes.io/ttl":"0","volumes.kubernetes.io/controller-managed-attach-detach":"true"}},"spec":{"podCIDR":"10.20.5.0/24"},"status":{"capacity":{"cpu":"4","ephemeral-storage":"79833Mi","hugepages-2Mi":"0","memory":"16296404Ki","pods":"200"},"allocatable":{"cpu":"4","ephemeral-storage":"75339870903","hugepages-2Mi":"0","memory":"16194004Ki","pods":"200"},"conditions":[{"type":"MemoryPressure","status":"False","lastHeartbeatTime":"2019-10-28T09:02:02Z","lastTransitionTime":"2019-10-28T08:30:42Z","reason":"KubeletHasSufficientMemory","message":"kubelet has sufficient memory available"},{"type":"DiskPressure","status":"False","lastHeartbeatTime":"2019-10-28T09:02:02Z","lastTransitionTime":"2019-10-28T08:30:42Z","reason":"KubeletHasNoDiskPressure","message":"kubelet has no disk pressure"},{"type":"PIDPressure","status":"False","lastHeartbeatTime":"2019-10-28T09:02:02Z","lastTransitionTime":"2019-10-28T08:30:42Z","reason":"KubeletHasSufficientPID","message":"kubelet has sufficient PID available"},{"type":"Ready","status":"True","lastHeartbeatTime":"2019-10-28T09:02:02Z","lastTransitionTime":"2019-10-28T08:30:42Z","reason":"KubeletReady","message":"kubelet is posting ready status"}],"addresses":[{"type":"InternalIP","address":"192.168.0.45"},{"type":"Hostname","address":"192.168.0.45"}],"daemonEndpoints":{"kubeletEndpoint":{"Port":10250}},"nodeInfo":{"machineID":"07a79cac7dcb4123a25f13c938dfc6f3","systemUUID":"032b0290-0434-05d8-3206-970700080009","bootID":"020b4ba4-6a0e-4968-a31e-837d03a1d71a","kernelVersion":"4.19.12-1.el7.elrepo.x86_64","osImage":"CentOS Linux 7 (Core)","containerRuntimeVersion":"docker://18.6.3","kubeletVersion":"v1.13.6","kubeProxyVersion":"v1.13.6","operatingSystem":"linux","architecture":"amd64"},"images":[{"names":["harbor.docker.zhiweireach.com/kubernetes/quay.io/external_storage/rbd-provisioner@sha256:b8ad79759bcd2dd289546bc350e3cf4cb2d7567ba9166e2ac57750a2277df4e9","quay.io/external_storage/rbd-provisioner@sha256:94fd36b8625141b62ff1addfa914d45f7b39619e55891bad0294263ecd2ce09a","harbor.docker.zhiweireach.com/kubernetes/quay.io/external_storage/rbd-provisioner:latest","quay.io/external_storage/rbd-provisioner:latest"],"sizeBytes":404591002},{"names":["registry.cn-hangzhou.aliyuncs.com/dnsjia/k8s@sha256:1d2e1229a918f4bc38b5a3f9f5f11302b3e71f8397b492afac7f273a0008776a","registry.cn-hangzhou.aliyuncs.com/dnsjia/k8s:kubernetes-dashboard-amd64_v1.10.0"],"sizeBytes":122460923},{"names":["coredns/coredns@sha256:12eb885b8685b1b13a04ecf5c23bc809c2e57917252fd7b0be9e9c00644e8ee5","coredns/coredns:1.6.2"],"sizeBytes":44100963},{"names":["gcr.azk8s.cn/google_containers/metrics-server-amd64@sha256:78938f933822856f443e6827fe5b37d6cc2f74ae888ac8b33d06fdbe5f8c658b","gcr.azk8s.cn/google_containers/metrics-server-amd64:v0.3.1"],"sizeBytes":40767713},{"names":["radial/busyboxplus@sha256:a68c05ab1112fd90ad7b14985a48520e9d26dbbe00cb9c09aa79fdc0ef46b372","radial/busyboxplus:curl"],"sizeBytes":4233788},{"names":["registry.cn-hangzhou.aliyuncs.com/google_containers/pause-amd64@sha256:759c3f0f6493093a9043cc813092290af69029699ade0e3dbe024e968fcb7cca","registry.cn-hangzhou.aliyuncs.com/google_containers/pause-amd64:3.1"],"sizeBytes":742472}]}},{"metadata":{"name":"192.168.0.46","selfLink":"/api/v1/nodes/192.168.0.46","uid":"0c957804-f3d9-11e9-a4a3-74d02bc70855","resourceVersion":"1940337","creationTimestamp":"2019-10-21T08:01:55Z","labels":{"beta.kubernetes.io/arch":"amd64","beta.kubernetes.io/os":"linux","edgenode":"true","kubernetes.io/hostname":"192.168.0.46","node-role.kubernetes.io/LB":"node46"},"annotations":{"node.alpha.kubernetes.io/ttl":"0","volumes.kubernetes.io/controller-managed-attach-detach":"true"}},"spec":{"podCIDR":"10.20.7.0/24","taints":[{"key":"edgenode","value":"true","effect":"NoSchedule"}]},"status":{"capacity":{"cpu":"4","ephemeral-storage":"84878856Ki","hugepages-2Mi":"0","memory":"32807696Ki","pods":"200"},"allocatable":{"cpu":"4","ephemeral-storage":"78224353561","hugepages-2Mi":"0","memory":"32705296Ki","pods":"200"},"conditions":[{"type":"MemoryPressure","status":"False","lastHeartbeatTime":"2019-10-28T09:02:09Z","lastTransitionTime":"2019-10-28T09:01:08Z","reason":"KubeletHasSufficientMemory","message":"kubelet has sufficient memory available"},{"type":"DiskPressure","status":"False","lastHeartbeatTime":"2019-10-28T09:02:09Z","lastTransitionTime":"2019-10-28T09:01:08Z","reason":"KubeletHasNoDiskPressure","message":"kubelet has no disk pressure"},{"type":"PIDPressure","status":"False","lastHeartbeatTime":"2019-10-28T09:02:09Z","lastTransitionTime":"2019-10-28T09:01:08Z","reason":"KubeletHasSufficientPID","message":"kubelet has sufficient PID available"},{"type":"Ready","status":"True","lastHeartbeatTime":"2019-10-28T09:02:09Z","lastTransitionTime":"2019-10-28T09:01:08Z","reason":"KubeletReady","message":"kubelet is posting ready status"}],"addresses":[{"type":"InternalIP","address":"192.168.0.46"},{"type":"Hostname","address":"192.168.0.46"}],"daemonEndpoints":{"kubeletEndpoint":{"Port":10250}},"nodeInfo":{"machineID":"ad0f2368b9b744b9b8c7e60e24e4d64a","systemUUID":"03de0294-0480-0502-a706-f30700080009","bootID":"1e7be4cb-ef6b-4be0-af94-d12c648b370d","kernelVersion":"4.19.12-1.el7.elrepo.x86_64","osImage":"CentOS Linux 7 (Core)","containerRuntimeVersion":"docker://18.6.3","kubeletVersion":"v1.13.6","kubeProxyVersion":"v1.13.6","operatingSystem":"linux","architecture":"amd64"},"images":[{"names":["traefik@sha256:20fd074599968976f5e06105a99b15de6d74d0b636ae5a8b760d81ee4c98f217","traefik:v1.7"],"sizeBytes":85586633},{"names":["registry.cn-hangzhou.aliyuncs.com/google_containers/pause-amd64@sha256:759c3f0f6493093a9043cc813092290af69029699ade0e3dbe024e968fcb7cca","registry.cn-hangzhou.aliyuncs.com/google_containers/pause-amd64:3.1"],"sizeBytes":742472}]}},{"metadata":{"name":"192.168.0.47","selfLink":"/api/v1/nodes/192.168.0.47","uid":"5e39cc7b-f182-11e9-84fc-902b34d396c8","resourceVersion":"1940326","creationTimestamp":"2019-10-18T08:36:24Z","labels":{"beta.kubernetes.io/arch":"amd64","beta.kubernetes.io/os":"linux","kubernetes.io/hostname":"192.168.0.47","node-role.kubernetes.io/node":"node47"},"annotations":{"node.alpha.kubernetes.io/ttl":"0","volumes.kubernetes.io/controller-managed-attach-detach":"true"}},"spec":{"podCIDR":"10.20.1.0/24"},"status":{"capacity":{"cpu":"4","ephemeral-storage":"92689184Ki","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"32810168Ki","pods":"200"},"allocatable":{"cpu":"4","ephemeral-storage":"85422351833","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"32707768Ki","pods":"200"},"conditions":[{"type":"MemoryPressure","status":"False","lastHeartbeatTime":"2019-10-28T09:02:03Z","lastTransitionTime":"2019-10-28T08:14:09Z","reason":"KubeletHasSufficientMemory","message":"kubelet has sufficient memory available"},{"type":"DiskPressure","status":"False","lastHeartbeatTime":"2019-10-28T09:02:03Z","lastTransitionTime":"2019-10-28T08:14:09Z","reason":"KubeletHasNoDiskPressure","message":"kubelet has no disk pressure"},{"type":"PIDPressure","status":"False","lastHeartbeatTime":"2019-10-28T09:02:03Z","lastTransitionTime":"2019-10-28T08:14:09Z","reason":"KubeletHasSufficientPID","message":"kubelet has sufficient PID available"},{"type":"Ready","status":"True","lastHeartbeatTime":"2019-10-28T09:02:03Z","lastTransitionTime":"2019-10-28T08:14:09Z","reason":"KubeletReady","message":"kubelet is posting ready status"}],"addresses":[{"type":"InternalIP","address":"192.168.0.47"},{"type":"Hostname","address":"192.168.0.47"}],"daemonEndpoints":{"kubeletEndpoint":{"Port":10250}},"nodeInfo":{"machineID":"fadf2695e72543ee939ec989564b0419","systemUUID":"f42d1b60-2302-11e6-9e84-3497f658b7c9","bootID":"1759ee65-42a5-4a8f-8f75-681995a0b6c9","kernelVersion":"4.19.12-1.el7.elrepo.x86_64","osImage":"CentOS Linux 7 (Core)","containerRuntimeVersion":"docker://18.6.3","kubeletVersion":"v1.13.6","kubeProxyVersion":"v1.13.6","operatingSystem":"linux","architecture":"amd64"},"images":[{"names":["coredns/coredns@sha256:12eb885b8685b1b13a04ecf5c23bc809c2e57917252fd7b0be9e9c00644e8ee5","coredns/coredns:1.6.2"],"sizeBytes":44100963},{"names":["gcr.azk8s.cn/google_containers/metrics-server-amd64@sha256:78938f933822856f443e6827fe5b37d6cc2f74ae888ac8b33d06fdbe5f8c658b","gcr.azk8s.cn/google_containers/metrics-server-amd64:v0.3.1"],"sizeBytes":40767713},{"names":["radial/busyboxplus@sha256:a68c05ab1112fd90ad7b14985a48520e9d26dbbe00cb9c09aa79fdc0ef46b372","radial/busyboxplus:curl"],"sizeBytes":4233788},{"names":["registry.cn-hangzhou.aliyuncs.com/google_containers/pause-amd64@sha256:759c3f0f6493093a9043cc813092290af69029699ade0e3dbe024e968fcb7cca","registry.cn-hangzhou.aliyuncs.com/google_containers/pause-amd64:3.1"],"sizeBytes":742472}]}},{"metadata":{"name":"192.168.0.49","selfLink":"/api/v1/nodes/192.168.0.49","uid":"b99a9542-f224-11e9-a4a3-74d02bc70855","resourceVersion":"1940329","creationTimestamp":"2019-10-19T03:58:35Z","labels":{"beta.kubernetes.io/arch":"amd64","beta.kubernetes.io/os":"linux","edgenode":"true","kubernetes.io/hostname":"192.168.0.49","node-role.kubernetes.io/LB":"node49"},"annotations":{"node.alpha.kubernetes.io/ttl":"0","volumes.kubernetes.io/controller-managed-attach-detach":"true"}},"spec":{"podCIDR":"10.20.6.0/24","taints":[{"key":"edgenode","value":"true","effect":"NoSchedule"}]},"status":{"capacity":{"cpu":"2","ephemeral-storage":"79833Mi","hugepages-2Mi":"0","memory":"16315300Ki","pods":"200"},"allocatable":{"cpu":"2","ephemeral-storage":"75339870903","hugepages-2Mi":"0","memory":"16212900Ki","pods":"200"},"conditions":[{"type":"MemoryPressure","status":"False","lastHeartbeatTime":"2019-10-28T09:02:04Z","lastTransitionTime":"2019-10-28T08:45:43Z","reason":"KubeletHasSufficientMemory","message":"kubelet has sufficient memory available"},{"type":"DiskPressure","status":"False","lastHeartbeatTime":"2019-10-28T09:02:04Z","lastTransitionTime":"2019-10-28T08:45:43Z","reason":"KubeletHasNoDiskPressure","message":"kubelet has no disk pressure"},{"type":"PIDPressure","status":"False","lastHeartbeatTime":"2019-10-28T09:02:04Z","lastTransitionTime":"2019-10-28T08:45:43Z","reason":"KubeletHasSufficientPID","message":"kubelet has sufficient PID available"},{"type":"Ready","status":"True","lastHeartbeatTime":"2019-10-28T09:02:04Z","lastTransitionTime":"2019-10-28T08:45:43Z","reason":"KubeletReady","message":"kubelet is posting ready status"}],"addresses":[{"type":"InternalIP","address":"192.168.0.49"},{"type":"Hostname","address":"192.168.0.49"}],"daemonEndpoints":{"kubeletEndpoint":{"Port":10250}},"nodeInfo":{"machineID":"c371d64c8cd64876b5ea8d919604a317","systemUUID":"03000200-0400-0500-0006-000700080009","bootID":"1ce6b758-e882-40a1-a79c-348b74b8a7d1","kernelVersion":"4.19.12-1.el7.elrepo.x86_64","osImage":"CentOS Linux 7 (Core)","containerRuntimeVersion":"docker://18.6.3","kubeletVersion":"v1.13.6","kubeProxyVersion":"v1.13.6","operatingSystem":"linux","architecture":"amd64"},"images":[{"names":["traefik@sha256:20fd074599968976f5e06105a99b15de6d74d0b636ae5a8b760d81ee4c98f217","traefik:v1.7"],"sizeBytes":85586633},{"names":["registry.cn-hangzhou.aliyuncs.com/google_containers/pause-amd64@sha256:759c3f0f6493093a9043cc813092290af69029699ade0e3dbe024e968fcb7cca","registry.cn-hangzhou.aliyuncs.com/google_containers/pause-amd64:3.1"],"sizeBytes":742472}]}},{"metadata":{"name":"192.168.0.51","selfLink":"/api/v1/nodes/192.168.0.51","uid":"e8ed1a4c-f3da-11e9-84fc-902b34d396c8","resourceVersion":"1940325","creationTimestamp":"2019-10-21T08:15:14Z","labels":{"beta.kubernetes.io/arch":"amd64","beta.kubernetes.io/os":"linux","edgenode":"true","kubernetes.io/hostname":"192.168.0.51","node-role.kubernetes.io/LB":"node51"},"annotations":{"node.alpha.kubernetes.io/ttl":"0","volumes.kubernetes.io/controller-managed-attach-detach":"true"}},"spec":{"podCIDR":"10.20.8.0/24","taints":[{"key":"edgenode","value":"true","effect":"NoSchedule"}]},"status":{"capacity":{"cpu":"4","ephemeral-storage":"82586216Ki","hugepages-2Mi":"0","memory":"32602700Ki","pods":"200"},"allocatable":{"cpu":"4","ephemeral-storage":"76111456540","hugepages-2Mi":"0","memory":"32500300Ki","pods":"200"},"conditions":[{"type":"MemoryPressure","status":"False","lastHeartbeatTime":"2019-10-28T09:02:02Z","lastTransitionTime":"2019-10-28T05:24:42Z","reason":"KubeletHasSufficientMemory","message":"kubelet has sufficient memory available"},{"type":"DiskPressure","status":"False","lastHeartbeatTime":"2019-10-28T09:02:02Z","lastTransitionTime":"2019-10-28T05:24:42Z","reason":"KubeletHasNoDiskPressure","message":"kubelet has no disk pressure"},{"type":"PIDPressure","status":"False","lastHeartbeatTime":"2019-10-28T09:02:02Z","lastTransitionTime":"2019-10-28T05:24:42Z","reason":"KubeletHasSufficientPID","message":"kubelet has sufficient PID available"},{"type":"Ready","status":"True","lastHeartbeatTime":"2019-10-28T09:02:02Z","lastTransitionTime":"2019-10-28T05:24:42Z","reason":"KubeletReady","message":"kubelet is posting ready status"}],"addresses":[{"type":"InternalIP","address":"192.168.0.51"},{"type":"Hostname","address":"192.168.0.51"}],"daemonEndpoints":{"kubeletEndpoint":{"Port":10250}},"nodeInfo":{"machineID":"0a7981f33ce6487da53328e0b39bc7aa","systemUUID":"4569c760-d83f-11dd-8dab-74d02bc70858","bootID":"4907b93c-41dc-42a4-a069-0a1f59b7d9e7","kernelVersion":"4.19.12-1.el7.elrepo.x86_64","osImage":"CentOS Linux 7 (Core)","containerRuntimeVersion":"docker://18.6.3","kubeletVersion":"v1.13.6","kubeProxyVersion":"v1.13.6","operatingSystem":"linux","architecture":"amd64"},"images":[{"names":["traefik@sha256:20fd074599968976f5e06105a99b15de6d74d0b636ae5a8b760d81ee4c98f217","traefik:v1.7"],"sizeBytes":85586633},{"names":["registry.cn-hangzhou.aliyuncs.com/google_containers/pause-amd64@sha256:759c3f0f6493093a9043cc813092290af69029699ade0e3dbe024e968fcb7cca","registry.cn-hangzhou.aliyuncs.com/google_containers/pause-amd64:3.1"],"sizeBytes":742472}]}}]}
NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
192.168.0.47 64m 1% 564Mi 1%
192.168.0.41 <unknown> <unknown> <unknown> <unknown>
192.168.0.44 <unknown> <unknown> <unknown> <unknown>
192.168.0.49 <unknown> <unknown> <unknown> <unknown>
192.168.0.40 <unknown> <unknown> <unknown> <unknown>
192.168.0.43 <unknown> <unknown> <unknown> <unknown>
192.168.0.45 <unknown> <unknown> <unknown> <unknown>
192.168.0.46 <unknown> <unknown> <unknown> <unknown>
192.168.0.51 <unknown> <unknown> <unknown> <unknown>
The result shows that I got CPU, memory and other information of all nodes, but only node47 node is displayed.
JokerDevops
on 28 Oct 2019
cczz12345
on 25 Nov 2019
Closing per Kubernetes issue triage policy
GitHub is not the right place for support requests.
If you're looking for help, check Stack Overflow and the troubleshooting guide.
You can also post your question on the Kubernetes Slack or the Discuss Kubernetes forum.
If the matter is security related, please disclose it privately via https://kubernetes.io/security/.
serathius
on 7 Feb 2020
Was this page helpful?
0 / 5 - 0 ratings
Related issues
shiveshabhishek
·
8Comments
nikhilkhanna
·
8Comments
floryut
·
8Comments
Rajat-0
·
5Comments
dudicoco
·
5Comments
Most helpful comment
@vvkkhjt try this flag --kubelet-insecure-tls