kind create cluster fails waiting for control-plane in AWS CodeBuild

Created on 4 Mar 2020 · 10Comments · Source: kubernetes-sigs/kind

What happened:
With kind 0.7.0, kind create cluster --wait 5m -v 1 fails after Waiting ≤ 5m0s for control-plane = Ready with: WARNING: Timed out waiting for Ready

Extending this to --wait 10m does not fix the issue.
Note that there is no problem with kind 0.6.1

What you expected to happen:
Cluster should be ready with message Ready after Xs

How to reproduce it (as minimally and precisely as possible):
Run a build project in AWS code build on the aws/codebuild/standard:2.0 environment image using the following minimal buildspec.yaml:

version: 0.2
phases:
  install:
    runtime-versions:
      docker: 18
      golang: 1.13
    commands:
      - apt-get update && apt-get install -y apt-transport-https
      - curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
      - echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | tee -a /etc/apt/sources.list.d/kubernetes.list
      - apt-get update
      - apt-get install -y kubectl
      - GO111MODULE="on" go get sigs.k8s.io/[email protected]
  pre_build:
    commands:
      - echo "Starting kubernetes cluster..."
      - kind create cluster --wait 5m -v 1
      - kubectl cluster-info --context kind-kind
    finally:
      - kubectl cluster-info dump
  build:
    commands:
      - echo "Created cluster"

Anything else we need to know?:
This works on kind <= 0.6.1 - issue only occurs with kind 0.7.0

Environment:

kind version: (use kind version): 0.7.0
Kubernetes version: (use kubectl version): 1.17.3
Docker version: (use docker info): 18 (unsure exactly which minor version is included in CodeBuild
OS (e.g. from /etc/os-release): aws/codebuild/standard:2.0 docker image (I believe this is Ubuntu 18.04)

help wanted kinbug prioritbacklog triagneeds-information

Source

MVJosh

👍1

Most helpful comment

Thanks for investigating - I don't know enough about CodeBuild to answer those questions unfortunately and the docs aren't too forthcoming about what goes on behind-the-scenes.

I'll look into building the CodeBuild image myself and reproducing locally, and will post back here once I've got some steps to reproduce locally without requiring an AWS account.

MVJosh on 6 Mar 2020

❤2

All 10 comments

Does codebuild have a proxy?

I do not use or have access to codebuild so I'll need more environment details.

BenTheElder on 5 Mar 2020

This wait is just looking for pods to be healthy on the control plane node. It should not have changed between versions, but the pods may not be healthy for some reason.

Can you export cluster logs (use --retain when creating and then kind export logs)?

BenTheElder on 5 Mar 2020

Does codebuild have a proxy?

I do not use or have access to codebuild so I'll need more environment details.

As far as I know there's no proxy, but I'm no expert on the internals of CodeBuild.

CodeBuild runs as an Ubuntu 18.04 docker container (with privileged access) - if you want to replicate locally you can build the image yourself from here: https://github.com/aws/aws-codebuild-docker-images/tree/master/ubuntu/standard/2.0 , spin up a container from the built image and then run the commands in the above Buildspec. This should reproduce the issue without having to run in AWS's hosted environment

MVJosh on 5 Mar 2020

This wait is just looking for pods to be healthy on the control plane node. It should not have changed between versions, but the pods may not be healthy for some reason.

Can you export cluster logs (use --retain when creating and then kind export logs)?

here's my kubelet.log:

-- Logs begin at Thu 2020-03-05 10:28:51 UTC, end at Thu 2020-03-05 10:31:51 UTC. --
Mar 05 10:28:51 kind-control-plane systemd[1]: Started kubelet: The Kubernetes Node Agent.
Mar 05 10:28:51 kind-control-plane kubelet[76]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 05 10:28:51 kind-control-plane kubelet[76]: F0305 10:28:51.165457      76 server.go:198] failed to load Kubelet config file /var/lib/kubelet/config.yaml, error failed to read kubelet config file "/var/lib/kubelet/config.yaml", error: open /var/lib/kubelet/config.yaml: no such file or directory
Mar 05 10:28:51 kind-control-plane systemd[1]: kubelet.service: Main process exited, code=exited, status=255/EXCEPTION
Mar 05 10:28:51 kind-control-plane systemd[1]: kubelet.service: Failed with result 'exit-code'.
Mar 05 10:28:51 kind-control-plane systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
Mar 05 10:28:51 kind-control-plane systemd[1]: Started kubelet: The Kubernetes Node Agent.
Mar 05 10:28:51 kind-control-plane kubelet[230]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 05 10:28:51 kind-control-plane kubelet[230]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 05 10:28:51 kind-control-plane kubelet[230]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 05 10:28:52 kind-control-plane kubelet[230]: I0305 10:28:52.255080     230 server.go:416] Version: v1.17.0
Mar 05 10:28:52 kind-control-plane kubelet[230]: I0305 10:28:52.255301     230 plugins.go:100] No cloud provider specified.
Mar 05 10:28:52 kind-control-plane kubelet[230]: I0305 10:28:52.255326     230 server.go:821] Client rotation is on, will bootstrap in background
Mar 05 10:28:52 kind-control-plane kubelet[230]: F0305 10:28:52.255374     230 server.go:273] failed to run Kubelet: unable to load bootstrap kubeconfig: stat /etc/kubernetes/bootstrap-kubelet.conf: no such file or directory
Mar 05 10:28:52 kind-control-plane systemd[1]: kubelet.service: Main process exited, code=exited, status=255/EXCEPTION
Mar 05 10:28:52 kind-control-plane systemd[1]: kubelet.service: Failed with result 'exit-code'.
Mar 05 10:28:53 kind-control-plane systemd[1]: kubelet.service: Service RestartSec=1s expired, scheduling restart.
Mar 05 10:28:53 kind-control-plane systemd[1]: kubelet.service: Scheduled restart job, restart counter is at 1.
Mar 05 10:28:53 kind-control-plane systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
Mar 05 10:28:53 kind-control-plane systemd[1]: Started kubelet: The Kubernetes Node Agent.
Mar 05 10:28:53 kind-control-plane kubelet[244]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 05 10:28:53 kind-control-plane kubelet[244]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 05 10:28:53 kind-control-plane kubelet[244]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 05 10:28:53 kind-control-plane kubelet[244]: I0305 10:28:53.375567     244 server.go:416] Version: v1.17.0
Mar 05 10:28:53 kind-control-plane kubelet[244]: I0305 10:28:53.375717     244 plugins.go:100] No cloud provider specified.
Mar 05 10:28:53 kind-control-plane kubelet[244]: I0305 10:28:53.375731     244 server.go:821] Client rotation is on, will bootstrap in background
Mar 05 10:28:53 kind-control-plane kubelet[244]: F0305 10:28:53.375759     244 server.go:273] failed to run Kubelet: unable to load bootstrap kubeconfig: stat /etc/kubernetes/bootstrap-kubelet.conf: no such file or directory
Mar 05 10:28:53 kind-control-plane systemd[1]: kubelet.service: Main process exited, code=exited, status=255/EXCEPTION
Mar 05 10:28:53 kind-control-plane systemd[1]: kubelet.service: Failed with result 'exit-code'.
Mar 05 10:28:54 kind-control-plane systemd[1]: kubelet.service: Service RestartSec=1s expired, scheduling restart.
Mar 05 10:28:54 kind-control-plane systemd[1]: kubelet.service: Scheduled restart job, restart counter is at 2.
Mar 05 10:28:54 kind-control-plane systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
Mar 05 10:28:54 kind-control-plane systemd[1]: Started kubelet: The Kubernetes Node Agent.
Mar 05 10:28:54 kind-control-plane kubelet[258]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 05 10:28:54 kind-control-plane kubelet[258]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 05 10:28:54 kind-control-plane kubelet[258]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 05 10:28:54 kind-control-plane kubelet[258]: I0305 10:28:54.625777     258 server.go:416] Version: v1.17.0
Mar 05 10:28:54 kind-control-plane kubelet[258]: I0305 10:28:54.625921     258 plugins.go:100] No cloud provider specified.
Mar 05 10:28:54 kind-control-plane kubelet[258]: I0305 10:28:54.625944     258 server.go:821] Client rotation is on, will bootstrap in background
Mar 05 10:28:54 kind-control-plane kubelet[258]: F0305 10:28:54.625974     258 server.go:273] failed to run Kubelet: unable to load bootstrap kubeconfig: stat /etc/kubernetes/bootstrap-kubelet.conf: no such file or directory
Mar 05 10:28:54 kind-control-plane systemd[1]: kubelet.service: Main process exited, code=exited, status=255/EXCEPTION
Mar 05 10:28:54 kind-control-plane systemd[1]: kubelet.service: Failed with result 'exit-code'.
Mar 05 10:28:55 kind-control-plane systemd[1]: kubelet.service: Service RestartSec=1s expired, scheduling restart.
Mar 05 10:28:55 kind-control-plane systemd[1]: kubelet.service: Scheduled restart job, restart counter is at 3.
Mar 05 10:28:55 kind-control-plane systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
Mar 05 10:28:55 kind-control-plane systemd[1]: Started kubelet: The Kubernetes Node Agent.
Mar 05 10:28:55 kind-control-plane kubelet[271]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 05 10:28:55 kind-control-plane kubelet[271]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 05 10:28:55 kind-control-plane kubelet[271]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 05 10:28:55 kind-control-plane kubelet[271]: I0305 10:28:55.875380     271 server.go:416] Version: v1.17.0
Mar 05 10:28:55 kind-control-plane kubelet[271]: I0305 10:28:55.875546     271 plugins.go:100] No cloud provider specified.
Mar 05 10:28:55 kind-control-plane kubelet[271]: I0305 10:28:55.875562     271 server.go:821] Client rotation is on, will bootstrap in background
Mar 05 10:28:55 kind-control-plane kubelet[271]: W0305 10:28:55.878601     271 container_manager_linux.go:843] CPUAccounting not enabled for pid: 271
Mar 05 10:28:55 kind-control-plane kubelet[271]: W0305 10:28:55.878618     271 container_manager_linux.go:846] MemoryAccounting not enabled for pid: 271
Mar 05 10:28:55 kind-control-plane kubelet[271]: E0305 10:28:55.888503     271 certificate_manager.go:421] Failed while requesting a signed certificate from the master: cannot create certificate signing request: Post https://172.18.0.2:6443/apis/certificates.k8s.io/v1beta1/certificatesigningrequests: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:28:58 kind-control-plane kubelet[271]: E0305 10:28:58.084560     271 certificate_manager.go:421] Failed while requesting a signed certificate from the master: cannot create certificate signing request: Post https://172.18.0.2:6443/apis/certificates.k8s.io/v1beta1/certificatesigningrequests: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:02 kind-control-plane kubelet[271]: E0305 10:29:02.421171     271 certificate_manager.go:421] Failed while requesting a signed certificate from the master: cannot create certificate signing request: Post https://172.18.0.2:6443/apis/certificates.k8s.io/v1beta1/certificatesigningrequests: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:10 kind-control-plane kubelet[271]: E0305 10:29:10.562151     271 certificate_manager.go:421] Failed while requesting a signed certificate from the master: cannot create certificate signing request: Post https://172.18.0.2:6443/apis/certificates.k8s.io/v1beta1/certificatesigningrequests: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:27 kind-control-plane kubelet[271]: E0305 10:29:27.088953     271 certificate_manager.go:421] Failed while requesting a signed certificate from the master: cannot create certificate signing request: Post https://172.18.0.2:6443/apis/certificates.k8s.io/v1beta1/certificatesigningrequests: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:27 kind-control-plane kubelet[271]: E0305 10:29:27.088980     271 certificate_manager.go:311] Reached backoff limit, still unable to rotate certs: timed out waiting for the condition
Mar 05 10:29:36 kind-control-plane kubelet[271]: I0305 10:29:36.471148     271 server.go:641] --cgroups-per-qos enabled, but --cgroup-root was not specified.  defaulting to /
Mar 05 10:29:36 kind-control-plane kubelet[271]: I0305 10:29:36.471641     271 container_manager_linux.go:265] container manager verified user specified cgroup-root exists: []
Mar 05 10:29:36 kind-control-plane kubelet[271]: I0305 10:29:36.471662     271 container_manager_linux.go:270] Creating Container Manager object based on Node Config: {RuntimeCgroupsName: SystemCgroupsName: KubeletCgroupsName: ContainerRuntime:remote CgroupsPerQOS:true CgroupRoot:/ CgroupDriver:cgroupfs KubeletRootDir:/var/lib/kubelet ProtectKernelDefaults:false NodeAllocatableConfig:{KubeReservedCgroupName: SystemReservedCgroupName: ReservedSystemCPUs: EnforceNodeAllocatable:map[pods:{}] KubeReserved:map[] SystemReserved:map[] HardEvictionThresholds:[]} QOSReserved:map[] ExperimentalCPUManagerPolicy:none ExperimentalCPUManagerReconcilePeriod:10s ExperimentalPodPidsLimit:-1 EnforceCPULimits:true CPUCFSQuotaPeriod:100ms ExperimentalTopologyManagerPolicy:none}
Mar 05 10:29:36 kind-control-plane kubelet[271]: I0305 10:29:36.471731     271 fake_topology_manager.go:29] [fake topologymanager] NewFakeManager
Mar 05 10:29:36 kind-control-plane kubelet[271]: I0305 10:29:36.471739     271 container_manager_linux.go:305] Creating device plugin manager: true
Mar 05 10:29:36 kind-control-plane kubelet[271]: I0305 10:29:36.471845     271 fake_topology_manager.go:39] [fake topologymanager] AddHintProvider HintProvider:  &{kubelet.sock /var/lib/kubelet/device-plugins/ map[] {0 0} <nil> {{} [0 0 0]} 0x1b084c0 0x3f47888 0x1b08d90 map[] map[] map[] map[] map[] 0xc0005865d0 [0] 0x3f47888}
Mar 05 10:29:36 kind-control-plane kubelet[271]: I0305 10:29:36.471886     271 state_mem.go:36] [cpumanager] initializing new in-memory state store
Mar 05 10:29:36 kind-control-plane kubelet[271]: I0305 10:29:36.474213     271 fake_topology_manager.go:39] [fake topologymanager] AddHintProvider HintProvider:  &{{0 0} 0x3f47888 10000000000 0xc0007da420 <nil> <nil> <nil> <nil> map[] 0x3f47888}
Mar 05 10:29:36 kind-control-plane kubelet[271]: I0305 10:29:36.474366     271 kubelet.go:286] Adding pod path: /etc/kubernetes/manifests
Mar 05 10:29:36 kind-control-plane kubelet[271]: I0305 10:29:36.474406     271 kubelet.go:311] Watching apiserver
Mar 05 10:29:36 kind-control-plane kubelet[271]: E0305 10:29:36.475007     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: Get https://172.18.0.2:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:36 kind-control-plane kubelet[271]: E0305 10:29:36.475024     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://172.18.0.2:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:36 kind-control-plane kubelet[271]: E0305 10:29:36.475114     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: Get https://172.18.0.2:6443/api/v1/nodes?fieldSelector=metadata.name%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:36 kind-control-plane kubelet[271]: W0305 10:29:36.475856     271 util_unix.go:103] Using "/run/containerd/containerd.sock" as endpoint is deprecated, please consider using full url format "unix:///run/containerd/containerd.sock".
Mar 05 10:29:36 kind-control-plane kubelet[271]: I0305 10:29:36.475917     271 remote_runtime.go:59] parsed scheme: ""
Mar 05 10:29:36 kind-control-plane kubelet[271]: I0305 10:29:36.475932     271 remote_runtime.go:59] scheme "" not registered, fallback to default scheme
Mar 05 10:29:36 kind-control-plane kubelet[271]: I0305 10:29:36.475963     271 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{/run/containerd/containerd.sock 0  <nil>}] <nil>}
Mar 05 10:29:36 kind-control-plane kubelet[271]: I0305 10:29:36.475973     271 clientconn.go:577] ClientConn switching balancer to "pick_first"
Mar 05 10:29:36 kind-control-plane kubelet[271]: W0305 10:29:36.476001     271 util_unix.go:103] Using "/run/containerd/containerd.sock" as endpoint is deprecated, please consider using full url format "unix:///run/containerd/containerd.sock".
Mar 05 10:29:36 kind-control-plane kubelet[271]: I0305 10:29:36.476018     271 remote_image.go:50] parsed scheme: ""
Mar 05 10:29:36 kind-control-plane kubelet[271]: I0305 10:29:36.476024     271 remote_image.go:50] scheme "" not registered, fallback to default scheme
Mar 05 10:29:36 kind-control-plane kubelet[271]: I0305 10:29:36.476035     271 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{/run/containerd/containerd.sock 0  <nil>}] <nil>}
Mar 05 10:29:36 kind-control-plane kubelet[271]: I0305 10:29:36.476040     271 clientconn.go:577] ClientConn switching balancer to "pick_first"
Mar 05 10:29:37 kind-control-plane kubelet[271]: E0305 10:29:37.475780     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: Get https://172.18.0.2:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:37 kind-control-plane kubelet[271]: E0305 10:29:37.476583     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://172.18.0.2:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:37 kind-control-plane kubelet[271]: E0305 10:29:37.477751     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: Get https://172.18.0.2:6443/api/v1/nodes?fieldSelector=metadata.name%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:38 kind-control-plane kubelet[271]: E0305 10:29:38.476261     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: Get https://172.18.0.2:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:38 kind-control-plane kubelet[271]: E0305 10:29:38.477475     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://172.18.0.2:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:38 kind-control-plane kubelet[271]: E0305 10:29:38.478652     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: Get https://172.18.0.2:6443/api/v1/nodes?fieldSelector=metadata.name%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:39 kind-control-plane kubelet[271]: E0305 10:29:39.476794     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: Get https://172.18.0.2:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:39 kind-control-plane kubelet[271]: E0305 10:29:39.477968     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://172.18.0.2:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:39 kind-control-plane kubelet[271]: E0305 10:29:39.479139     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: Get https://172.18.0.2:6443/api/v1/nodes?fieldSelector=metadata.name%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:40 kind-control-plane kubelet[271]: E0305 10:29:40.477565     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: Get https://172.18.0.2:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:40 kind-control-plane kubelet[271]: E0305 10:29:40.478392     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://172.18.0.2:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:40 kind-control-plane kubelet[271]: E0305 10:29:40.479832     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: Get https://172.18.0.2:6443/api/v1/nodes?fieldSelector=metadata.name%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:41 kind-control-plane kubelet[271]: E0305 10:29:41.478114     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: Get https://172.18.0.2:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:41 kind-control-plane kubelet[271]: E0305 10:29:41.479339     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://172.18.0.2:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:41 kind-control-plane kubelet[271]: E0305 10:29:41.480378     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: Get https://172.18.0.2:6443/api/v1/nodes?fieldSelector=metadata.name%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:42 kind-control-plane kubelet[271]: E0305 10:29:42.478601     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: Get https://172.18.0.2:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:42 kind-control-plane kubelet[271]: E0305 10:29:42.479758     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://172.18.0.2:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:42 kind-control-plane kubelet[271]: E0305 10:29:42.480898     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: Get https://172.18.0.2:6443/api/v1/nodes?fieldSelector=metadata.name%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:43 kind-control-plane kubelet[271]: E0305 10:29:43.479329     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: Get https://172.18.0.2:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:43 kind-control-plane kubelet[271]: E0305 10:29:43.480192     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://172.18.0.2:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:43 kind-control-plane kubelet[271]: E0305 10:29:43.481423     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: Get https://172.18.0.2:6443/api/v1/nodes?fieldSelector=metadata.name%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:44 kind-control-plane kubelet[271]: E0305 10:29:44.480045     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: Get https://172.18.0.2:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:44 kind-control-plane kubelet[271]: E0305 10:29:44.481147     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://172.18.0.2:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:44 kind-control-plane kubelet[271]: E0305 10:29:44.482231     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: Get https://172.18.0.2:6443/api/v1/nodes?fieldSelector=metadata.name%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:45 kind-control-plane kubelet[271]: E0305 10:29:45.480701     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: Get https://172.18.0.2:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:45 kind-control-plane kubelet[271]: E0305 10:29:45.481625     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://172.18.0.2:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:45 kind-control-plane kubelet[271]: E0305 10:29:45.482752     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: Get https://172.18.0.2:6443/api/v1/nodes?fieldSelector=metadata.name%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:46 kind-control-plane kubelet[271]: E0305 10:29:46.481434     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: Get https://172.18.0.2:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:46 kind-control-plane kubelet[271]: E0305 10:29:46.482545     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://172.18.0.2:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:46 kind-control-plane kubelet[271]: E0305 10:29:46.483650     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: Get https://172.18.0.2:6443/api/v1/nodes?fieldSelector=metadata.name%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:47 kind-control-plane kubelet[271]: E0305 10:29:47.482146     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: Get https://172.18.0.2:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:47 kind-control-plane kubelet[271]: E0305 10:29:47.482944     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://172.18.0.2:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:47 kind-control-plane kubelet[271]: E0305 10:29:47.484255     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: Get https://172.18.0.2:6443/api/v1/nodes?fieldSelector=metadata.name%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:48 kind-control-plane kubelet[271]: E0305 10:29:48.482877     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: Get https://172.18.0.2:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:48 kind-control-plane kubelet[271]: E0305 10:29:48.483993     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://172.18.0.2:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:48 kind-control-plane kubelet[271]: E0305 10:29:48.484826     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: Get https://172.18.0.2:6443/api/v1/nodes?fieldSelector=metadata.name%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:49 kind-control-plane kubelet[271]: E0305 10:29:49.483480     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: Get https://172.18.0.2:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:49 kind-control-plane kubelet[271]: E0305 10:29:49.484565     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://172.18.0.2:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:49 kind-control-plane kubelet[271]: E0305 10:29:49.485668     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: Get https://172.18.0.2:6443/api/v1/nodes?fieldSelector=metadata.name%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:50 kind-control-plane kubelet[271]: E0305 10:29:50.484162     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: Get https://172.18.0.2:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:50 kind-control-plane kubelet[271]: E0305 10:29:50.484999     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://172.18.0.2:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:50 kind-control-plane kubelet[271]: E0305 10:29:50.486351     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: Get https://172.18.0.2:6443/api/v1/nodes?fieldSelector=metadata.name%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:51 kind-control-plane kubelet[271]: E0305 10:29:51.484959     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: Get https://172.18.0.2:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:51 kind-control-plane kubelet[271]: E0305 10:29:51.485815     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://172.18.0.2:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:51 kind-control-plane kubelet[271]: E0305 10:29:51.487007     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: Get https://172.18.0.2:6443/api/v1/nodes?fieldSelector=metadata.name%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:52 kind-control-plane kubelet[271]: E0305 10:29:52.485625     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: Get https://172.18.0.2:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:52 kind-control-plane kubelet[271]: E0305 10:29:52.486721     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://172.18.0.2:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:52 kind-control-plane kubelet[271]: E0305 10:29:52.487881     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: Get https://172.18.0.2:6443/api/v1/nodes?fieldSelector=metadata.name%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:53 kind-control-plane kubelet[271]: E0305 10:29:53.486272     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: Get https://172.18.0.2:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:53 kind-control-plane kubelet[271]: E0305 10:29:53.487159     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://172.18.0.2:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:53 kind-control-plane kubelet[271]: E0305 10:29:53.488559     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: Get https://172.18.0.2:6443/api/v1/nodes?fieldSelector=metadata.name%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:54 kind-control-plane kubelet[271]: E0305 10:29:54.486813     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: Get https://172.18.0.2:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:54 kind-control-plane kubelet[271]: E0305 10:29:54.488092     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://172.18.0.2:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:54 kind-control-plane kubelet[271]: E0305 10:29:54.489289     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: Get https://172.18.0.2:6443/api/v1/nodes?fieldSelector=metadata.name%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:55 kind-control-plane kubelet[271]: E0305 10:29:55.487561     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: Get https://172.18.0.2:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:55 kind-control-plane kubelet[271]: E0305 10:29:55.488543     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://172.18.0.2:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:55 kind-control-plane kubelet[271]: E0305 10:29:55.489775     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: Get https://172.18.0.2:6443/api/v1/nodes?fieldSelector=metadata.name%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:56 kind-control-plane kubelet[271]: E0305 10:29:56.488185     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: Get https://172.18.0.2:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:56 kind-control-plane kubelet[271]: E0305 10:29:56.489195     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://172.18.0.2:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:56 kind-control-plane kubelet[271]: E0305 10:29:56.490263     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: Get https://172.18.0.2:6443/api/v1/nodes?fieldSelector=metadata.name%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:56 kind-control-plane kubelet[271]: E0305 10:29:56.811059     271 aws_credentials.go:77] while getting AWS credentials NoCredentialProviders: no valid providers in chain. Deprecated.
Mar 05 10:29:56 kind-control-plane kubelet[271]:         For verbose messaging see aws.Config.CredentialsChainVerboseErrors
Mar 05 10:29:56 kind-control-plane kubelet[271]: I0305 10:29:56.811908     271 kuberuntime_manager.go:211] Container runtime containerd initialized, version: v1.3.2, apiVersion: v1alpha2
Mar 05 10:29:56 kind-control-plane kubelet[271]: W0305 10:29:56.812102     271 probe.go:268] Flexvolume plugin directory at /usr/libexec/kubernetes/kubelet-plugins/volume/exec/ does not exist. Recreating.
Mar 05 10:29:56 kind-control-plane kubelet[271]: I0305 10:29:56.812448     271 server.go:1113] Started kubelet
Mar 05 10:29:56 kind-control-plane kubelet[271]: I0305 10:29:56.812547     271 server.go:143] Starting to listen on 0.0.0.0:10250
Mar 05 10:29:56 kind-control-plane kubelet[271]: E0305 10:29:56.812835     271 event.go:272] Unable to write event: 'Post https://172.18.0.2:6443/api/v1/namespaces/default/events: dial tcp 172.18.0.2:6443: connect: connection refused' (may retry after sleeping)
Mar 05 10:29:56 kind-control-plane kubelet[271]: I0305 10:29:56.813985     271 fs_resource_analyzer.go:64] Starting FS ResourceAnalyzer
Mar 05 10:29:56 kind-control-plane kubelet[271]: I0305 10:29:56.814152     271 server.go:354] Adding debug handlers to kubelet server.
Mar 05 10:29:56 kind-control-plane kubelet[271]: I0305 10:29:56.814188     271 volume_manager.go:265] Starting Kubelet Volume Manager
Mar 05 10:29:56 kind-control-plane kubelet[271]: I0305 10:29:56.814385     271 desired_state_of_world_populator.go:138] Desired state populator starts to run
Mar 05 10:29:56 kind-control-plane kubelet[271]: E0305 10:29:56.815074     271 reflector.go:156] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: Get https://172.18.0.2:6443/apis/storage.k8s.io/v1beta1/csidrivers?limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:56 kind-control-plane kubelet[271]: E0305 10:29:56.815729     271 controller.go:135] failed to ensure node lease exists, will retry in 200ms, error: Get https://172.18.0.2:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/kind-control-plane?timeout=10s: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:56 kind-control-plane kubelet[271]: E0305 10:29:56.815795     271 kubelet.go:2183] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized
Mar 05 10:29:56 kind-control-plane kubelet[271]: I0305 10:29:56.816353     271 clientconn.go:104] parsed scheme: "unix"
Mar 05 10:29:56 kind-control-plane kubelet[271]: I0305 10:29:56.816529     271 clientconn.go:104] scheme "unix" not registered, fallback to default scheme
Mar 05 10:29:56 kind-control-plane kubelet[271]: I0305 10:29:56.816633     271 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock 0  <nil>}] <nil>}
Mar 05 10:29:56 kind-control-plane kubelet[271]: I0305 10:29:56.816663     271 clientconn.go:577] ClientConn switching balancer to "pick_first"
Mar 05 10:29:56 kind-control-plane kubelet[271]: I0305 10:29:56.845777     271 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Mar 05 10:29:56 kind-control-plane kubelet[271]: I0305 10:29:56.846404     271 cpu_manager.go:173] [cpumanager] starting with none policy
Mar 05 10:29:56 kind-control-plane kubelet[271]: I0305 10:29:56.846418     271 cpu_manager.go:174] [cpumanager] reconciling every 10s
Mar 05 10:29:56 kind-control-plane kubelet[271]: I0305 10:29:56.846425     271 policy_none.go:43] [cpumanager] none policy: Start
Mar 05 10:29:56 kind-control-plane kubelet[271]: E0305 10:29:56.851479     271 cgroup_manager_linux.go:488] cgroup update failed failed to set supported cgroup subsystems for cgroup [kubepods]: failed to find subsystem mount for required subsystem: pids
Mar 05 10:29:56 kind-control-plane kubelet[271]: E0305 10:29:56.855962     271 cgroup_manager_linux.go:488] cgroup update failed failed to set supported cgroup subsystems for cgroup [kubepods burstable]: failed to find subsystem mount for required subsystem: pids
Mar 05 10:29:56 kind-control-plane kubelet[271]: E0305 10:29:56.860381     271 cgroup_manager_linux.go:488] cgroup update failed failed to set supported cgroup subsystems for cgroup [kubepods besteffort]: failed to find subsystem mount for required subsystem: pids
Mar 05 10:29:56 kind-control-plane kubelet[271]: W0305 10:29:56.860441     271 manager.go:577] Failed to retrieve checkpoint for "kubelet_internal_checkpoint": checkpoint is not found
Mar 05 10:29:56 kind-control-plane kubelet[271]: I0305 10:29:56.860610     271 plugin_manager.go:114] Starting Kubelet Plugin Manager
Mar 05 10:29:56 kind-control-plane kubelet[271]: W0305 10:29:56.860737     271 container_manager_linux.go:843] CPUAccounting not enabled for pid: 271
Mar 05 10:29:56 kind-control-plane kubelet[271]: W0305 10:29:56.860760     271 container_manager_linux.go:846] MemoryAccounting not enabled for pid: 271
Mar 05 10:29:56 kind-control-plane kubelet[271]: E0305 10:29:56.860936     271 qos_container_manager_linux.go:328] [ContainerManager]: Failed to update QoS cgroup configuration
Mar 05 10:29:56 kind-control-plane kubelet[271]: W0305 10:29:56.860956     271 qos_container_manager_linux.go:138] [ContainerManager] Failed to reserve QoS requests: failed to set supported cgroup subsystems for cgroup [kubepods besteffort]: failed to find subsystem mount for required subsystem: pids
Mar 05 10:29:56 kind-control-plane kubelet[271]: E0305 10:29:56.860944     271 eviction_manager.go:246] eviction manager: failed to get summary stats: failed to get node info: node "kind-control-plane" not found
Mar 05 10:29:56 kind-control-plane kubelet[271]: I0305 10:29:56.914314     271 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Mar 05 10:29:56 kind-control-plane kubelet[271]: E0305 10:29:56.914341     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:29:56 kind-control-plane kubelet[271]: I0305 10:29:56.914916     271 kubelet_node_status.go:70] Attempting to register node kind-control-plane
Mar 05 10:29:56 kind-control-plane kubelet[271]: E0305 10:29:56.915146     271 kubelet_node_status.go:92] Unable to register node "kind-control-plane" with API server: Post https://172.18.0.2:6443/api/v1/nodes: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:57 kind-control-plane kubelet[271]: E0305 10:29:57.014548     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:29:57 kind-control-plane kubelet[271]: E0305 10:29:57.016150     271 controller.go:135] failed to ensure node lease exists, will retry in 400ms, error: Get https://172.18.0.2:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/kind-control-plane?timeout=10s: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:57 kind-control-plane kubelet[271]: E0305 10:29:57.114766     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:29:57 kind-control-plane kubelet[271]: I0305 10:29:57.115308     271 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Mar 05 10:29:57 kind-control-plane kubelet[271]: I0305 10:29:57.115880     271 kubelet_node_status.go:70] Attempting to register node kind-control-plane
Mar 05 10:29:57 kind-control-plane kubelet[271]: E0305 10:29:57.116082     271 kubelet_node_status.go:92] Unable to register node "kind-control-plane" with API server: Post https://172.18.0.2:6443/api/v1/nodes: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:57 kind-control-plane kubelet[271]: I0305 10:29:57.129294     271 status_manager.go:157] Starting to sync pod status with apiserver
Mar 05 10:29:57 kind-control-plane kubelet[271]: I0305 10:29:57.129323     271 kubelet.go:1820] Starting kubelet main sync loop.
Mar 05 10:29:57 kind-control-plane kubelet[271]: E0305 10:29:57.129373     271 kubelet.go:1844] skipping pod synchronization - PLEG is not healthy: pleg has yet to be successful
Mar 05 10:29:57 kind-control-plane kubelet[271]: E0305 10:29:57.129857     271 reflector.go:156] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: Get https://172.18.0.2:6443/apis/node.k8s.io/v1beta1/runtimeclasses?limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:57 kind-control-plane kubelet[271]: E0305 10:29:57.133552     271 csi_plugin.go:267] Failed to initialize CSINodeInfo: error updating CSINode annotation: timed out waiting for the condition; caused by: Get https://172.18.0.2:6443/apis/storage.k8s.io/v1/csinodes/kind-control-plane: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:57 kind-control-plane kubelet[271]: E0305 10:29:57.214983     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:29:57 kind-control-plane kubelet[271]: I0305 10:29:57.229715     271 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Mar 05 10:29:57 kind-control-plane kubelet[271]: I0305 10:29:57.233389     271 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Mar 05 10:29:57 kind-control-plane kubelet[271]: I0305 10:29:57.233476     271 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Mar 05 10:29:57 kind-control-plane kubelet[271]: E0305 10:29:57.234250     271 qos_container_manager_linux.go:328] [ContainerManager]: Failed to update QoS cgroup configuration
Mar 05 10:29:57 kind-control-plane kubelet[271]: I0305 10:29:57.236185     271 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Mar 05 10:29:57 kind-control-plane kubelet[271]: I0305 10:29:57.236235     271 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Mar 05 10:29:57 kind-control-plane kubelet[271]: E0305 10:29:57.237609     271 qos_container_manager_linux.go:328] [ContainerManager]: Failed to update QoS cgroup configuration
Mar 05 10:29:57 kind-control-plane kubelet[271]: I0305 10:29:57.239751     271 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Mar 05 10:29:57 kind-control-plane kubelet[271]: I0305 10:29:57.239923     271 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Mar 05 10:29:57 kind-control-plane kubelet[271]: E0305 10:29:57.241106     271 qos_container_manager_linux.go:328] [ContainerManager]: Failed to update QoS cgroup configuration
Mar 05 10:29:57 kind-control-plane kubelet[271]: E0305 10:29:57.241789     271 cgroup_manager_linux.go:488] cgroup update failed failed to set supported cgroup subsystems for cgroup [kubepods burstable podeec632247bbda1d470b7e95bacb0a8fb]: failed to find subsystem mount for required subsystem: pids
Mar 05 10:29:57 kind-control-plane kubelet[271]: I0305 10:29:57.243031     271 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Mar 05 10:29:57 kind-control-plane kubelet[271]: E0305 10:29:57.243861     271 qos_container_manager_linux.go:328] [ContainerManager]: Failed to update QoS cgroup configuration
Mar 05 10:29:57 kind-control-plane kubelet[271]: E0305 10:29:57.244744     271 cgroup_manager_linux.go:488] cgroup update failed failed to set supported cgroup subsystems for cgroup [kubepods burstable pod15e79e27a50d92dc481a5aaaad4399d8]: failed to find subsystem mount for required subsystem: pids
Mar 05 10:29:57 kind-control-plane kubelet[271]: E0305 10:29:57.246619     271 cgroup_manager_linux.go:488] cgroup update failed failed to set supported cgroup subsystems for cgroup [kubepods burstable podff67867321338ffd885039e188f6b424]: failed to find subsystem mount for required subsystem: pids
Mar 05 10:29:57 kind-control-plane kubelet[271]: E0305 10:29:57.249346     271 cgroup_manager_linux.go:488] cgroup update failed failed to set supported cgroup subsystems for cgroup [kubepods besteffort pod38b67f836febc1aa47393a3432e1a094]: failed to find subsystem mount for required subsystem: pids
Mar 05 10:29:57 kind-control-plane kubelet[271]: E0305 10:29:57.315166     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:29:57 kind-control-plane kubelet[271]: I0305 10:29:57.315293     271 reconciler.go:209] operationExecutor.VerifyControllerAttachedVolume started for volume "etcd-certs" (UniqueName: "kubernetes.io/host-path/38b67f836febc1aa47393a3432e1a094-etcd-certs") pod "etcd-kind-control-plane" (UID: "38b67f836febc1aa47393a3432e1a094")
Mar 05 10:29:57 kind-control-plane kubelet[271]: I0305 10:29:57.315326     271 reconciler.go:209] operationExecutor.VerifyControllerAttachedVolume started for volume "ca-certs" (UniqueName: "kubernetes.io/host-path/eec632247bbda1d470b7e95bacb0a8fb-ca-certs") pod "kube-apiserver-kind-control-plane" (UID: "eec632247bbda1d470b7e95bacb0a8fb")
Mar 05 10:29:57 kind-control-plane kubelet[271]: I0305 10:29:57.315347     271 reconciler.go:209] operationExecutor.VerifyControllerAttachedVolume started for volume "etc-ca-certificates" (UniqueName: "kubernetes.io/host-path/15e79e27a50d92dc481a5aaaad4399d8-etc-ca-certificates") pod "kube-controller-manager-kind-control-plane" (UID: "15e79e27a50d92dc481a5aaaad4399d8")
Mar 05 10:29:57 kind-control-plane kubelet[271]: I0305 10:29:57.315367     271 reconciler.go:209] operationExecutor.VerifyControllerAttachedVolume started for volume "usr-share-ca-certificates" (UniqueName: "kubernetes.io/host-path/eec632247bbda1d470b7e95bacb0a8fb-usr-share-ca-certificates") pod "kube-apiserver-kind-control-plane" (UID: "eec632247bbda1d470b7e95bacb0a8fb")
Mar 05 10:29:57 kind-control-plane kubelet[271]: I0305 10:29:57.315387     271 reconciler.go:209] operationExecutor.VerifyControllerAttachedVolume started for volume "ca-certs" (UniqueName: "kubernetes.io/host-path/15e79e27a50d92dc481a5aaaad4399d8-ca-certs") pod "kube-controller-manager-kind-control-plane" (UID: "15e79e27a50d92dc481a5aaaad4399d8")
Mar 05 10:29:57 kind-control-plane kubelet[271]: I0305 10:29:57.315408     271 reconciler.go:209] operationExecutor.VerifyControllerAttachedVolume started for volume "flexvolume-dir" (UniqueName: "kubernetes.io/host-path/15e79e27a50d92dc481a5aaaad4399d8-flexvolume-dir") pod "kube-controller-manager-kind-control-plane" (UID: "15e79e27a50d92dc481a5aaaad4399d8")
Mar 05 10:29:57 kind-control-plane kubelet[271]: I0305 10:29:57.315430     271 reconciler.go:209] operationExecutor.VerifyControllerAttachedVolume started for volume "usr-local-share-ca-certificates" (UniqueName: "kubernetes.io/host-path/15e79e27a50d92dc481a5aaaad4399d8-usr-local-share-ca-certificates") pod "kube-controller-manager-kind-control-plane" (UID: "15e79e27a50d92dc481a5aaaad4399d8")
Mar 05 10:29:57 kind-control-plane kubelet[271]: I0305 10:29:57.315463     271 reconciler.go:209] operationExecutor.VerifyControllerAttachedVolume started for volume "etc-ca-certificates" (UniqueName: "kubernetes.io/host-path/eec632247bbda1d470b7e95bacb0a8fb-etc-ca-certificates") pod "kube-apiserver-kind-control-plane" (UID: "eec632247bbda1d470b7e95bacb0a8fb")
Mar 05 10:29:57 kind-control-plane kubelet[271]: I0305 10:29:57.315516     271 reconciler.go:209] operationExecutor.VerifyControllerAttachedVolume started for volume "usr-local-share-ca-certificates" (UniqueName: "kubernetes.io/host-path/eec632247bbda1d470b7e95bacb0a8fb-usr-local-share-ca-certificates") pod "kube-apiserver-kind-control-plane" (UID: "eec632247bbda1d470b7e95bacb0a8fb")
Mar 05 10:29:57 kind-control-plane kubelet[271]: I0305 10:29:57.315547     271 reconciler.go:209] operationExecutor.VerifyControllerAttachedVolume started for volume "kubeconfig" (UniqueName: "kubernetes.io/host-path/15e79e27a50d92dc481a5aaaad4399d8-kubeconfig") pod "kube-controller-manager-kind-control-plane" (UID: "15e79e27a50d92dc481a5aaaad4399d8")
Mar 05 10:29:57 kind-control-plane kubelet[271]: I0305 10:29:57.315569     271 reconciler.go:209] operationExecutor.VerifyControllerAttachedVolume started for volume "usr-share-ca-certificates" (UniqueName: "kubernetes.io/host-path/15e79e27a50d92dc481a5aaaad4399d8-usr-share-ca-certificates") pod "kube-controller-manager-kind-control-plane" (UID: "15e79e27a50d92dc481a5aaaad4399d8")
Mar 05 10:29:57 kind-control-plane kubelet[271]: I0305 10:29:57.315615     271 reconciler.go:209] operationExecutor.VerifyControllerAttachedVolume started for volume "etcd-data" (UniqueName: "kubernetes.io/host-path/38b67f836febc1aa47393a3432e1a094-etcd-data") pod "etcd-kind-control-plane" (UID: "38b67f836febc1aa47393a3432e1a094")
Mar 05 10:29:57 kind-control-plane kubelet[271]: I0305 10:29:57.315689     271 reconciler.go:209] operationExecutor.VerifyControllerAttachedVolume started for volume "k8s-certs" (UniqueName: "kubernetes.io/host-path/eec632247bbda1d470b7e95bacb0a8fb-k8s-certs") pod "kube-apiserver-kind-control-plane" (UID: "eec632247bbda1d470b7e95bacb0a8fb")
Mar 05 10:29:57 kind-control-plane kubelet[271]: I0305 10:29:57.315711     271 reconciler.go:209] operationExecutor.VerifyControllerAttachedVolume started for volume "k8s-certs" (UniqueName: "kubernetes.io/host-path/15e79e27a50d92dc481a5aaaad4399d8-k8s-certs") pod "kube-controller-manager-kind-control-plane" (UID: "15e79e27a50d92dc481a5aaaad4399d8")
Mar 05 10:29:57 kind-control-plane kubelet[271]: E0305 10:29:57.415413     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:29:57 kind-control-plane kubelet[271]: I0305 10:29:57.416082     271 reconciler.go:209] operationExecutor.VerifyControllerAttachedVolume started for volume "kubeconfig" (UniqueName: "kubernetes.io/host-path/ff67867321338ffd885039e188f6b424-kubeconfig") pod "kube-scheduler-kind-control-plane" (UID: "ff67867321338ffd885039e188f6b424")
Mar 05 10:29:57 kind-control-plane kubelet[271]: E0305 10:29:57.416589     271 controller.go:135] failed to ensure node lease exists, will retry in 800ms, error: Get https://172.18.0.2:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/kind-control-plane?timeout=10s: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:57 kind-control-plane kubelet[271]: W0305 10:29:57.488352     271 status_manager.go:530] Failed to get status for pod "kube-apiserver-kind-control-plane_kube-system(eec632247bbda1d470b7e95bacb0a8fb)": Get https://172.18.0.2:6443/api/v1/namespaces/kube-system/pods/kube-apiserver-kind-control-plane: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:57 kind-control-plane kubelet[271]: E0305 10:29:57.515670     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:29:57 kind-control-plane kubelet[271]: I0305 10:29:57.516291     271 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Mar 05 10:29:57 kind-control-plane kubelet[271]: I0305 10:29:57.516848     271 kubelet_node_status.go:70] Attempting to register node kind-control-plane
Mar 05 10:29:57 kind-control-plane kubelet[271]: E0305 10:29:57.615944     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:29:57 kind-control-plane kubelet[271]: W0305 10:29:57.688425     271 status_manager.go:530] Failed to get status for pod "kube-controller-manager-kind-control-plane_kube-system(15e79e27a50d92dc481a5aaaad4399d8)": Get https://172.18.0.2:6443/api/v1/namespaces/kube-system/pods/kube-controller-manager-kind-control-plane: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:57 kind-control-plane kubelet[271]: E0305 10:29:57.716196     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:29:57 kind-control-plane kubelet[271]: E0305 10:29:57.816432     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:29:57 kind-control-plane kubelet[271]: E0305 10:29:57.888670     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: Get https://172.18.0.2:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:57 kind-control-plane kubelet[271]: E0305 10:29:57.916628     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:29:58 kind-control-plane kubelet[271]: E0305 10:29:58.016855     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:29:58 kind-control-plane kubelet[271]: E0305 10:29:58.088548     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://172.18.0.2:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:58 kind-control-plane kubelet[271]: E0305 10:29:58.117147     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:29:58 kind-control-plane kubelet[271]: E0305 10:29:58.217285     271 controller.go:135] failed to ensure node lease exists, will retry in 1.6s, error: Get https://172.18.0.2:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/kind-control-plane?timeout=10s: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:58 kind-control-plane kubelet[271]: E0305 10:29:58.217365     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:29:58 kind-control-plane kubelet[271]: E0305 10:29:58.288685     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: Get https://172.18.0.2:6443/api/v1/nodes?fieldSelector=metadata.name%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:58 kind-control-plane kubelet[271]: E0305 10:29:58.317544     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:29:58 kind-control-plane kubelet[271]: E0305 10:29:58.417770     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:29:58 kind-control-plane kubelet[271]: E0305 10:29:58.488557     271 kubelet_node_status.go:92] Unable to register node "kind-control-plane" with API server: Post https://172.18.0.2:6443/api/v1/nodes: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:58 kind-control-plane kubelet[271]: E0305 10:29:58.518031     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:29:58 kind-control-plane kubelet[271]: E0305 10:29:58.618271     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:29:58 kind-control-plane kubelet[271]: E0305 10:29:58.688477     271 csi_plugin.go:267] Failed to initialize CSINodeInfo: error updating CSINode annotation: timed out waiting for the condition; caused by: Get https://172.18.0.2:6443/apis/storage.k8s.io/v1/csinodes/kind-control-plane: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:58 kind-control-plane kubelet[271]: E0305 10:29:58.718500     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:29:58 kind-control-plane kubelet[271]: E0305 10:29:58.818727     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:29:58 kind-control-plane kubelet[271]: W0305 10:29:58.888444     271 status_manager.go:530] Failed to get status for pod "kube-scheduler-kind-control-plane_kube-system(ff67867321338ffd885039e188f6b424)": Get https://172.18.0.2:6443/api/v1/namespaces/kube-system/pods/kube-scheduler-kind-control-plane: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:58 kind-control-plane kubelet[271]: E0305 10:29:58.918945     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:29:59 kind-control-plane kubelet[271]: E0305 10:29:59.019153     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:29:59 kind-control-plane kubelet[271]: E0305 10:29:59.088539     271 reflector.go:156] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: Get https://172.18.0.2:6443/apis/storage.k8s.io/v1beta1/csidrivers?limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:59 kind-control-plane kubelet[271]: E0305 10:29:59.091238     271 certificate_manager.go:421] Failed while requesting a signed certificate from the master: cannot create certificate signing request: Post https://172.18.0.2:6443/apis/certificates.k8s.io/v1beta1/certificatesigningrequests: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:59 kind-control-plane kubelet[271]: E0305 10:29:59.119355     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:29:59 kind-control-plane kubelet[271]: E0305 10:29:59.219564     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:29:59 kind-control-plane kubelet[271]: E0305 10:29:59.288496     271 reflector.go:156] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: Get https://172.18.0.2:6443/apis/node.k8s.io/v1beta1/runtimeclasses?limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:59 kind-control-plane kubelet[271]: I0305 10:29:59.288806     271 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Mar 05 10:29:59 kind-control-plane kubelet[271]: I0305 10:29:59.289664     271 kubelet_node_status.go:70] Attempting to register node kind-control-plane
Mar 05 10:29:59 kind-control-plane kubelet[271]: E0305 10:29:59.319785     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:29:59 kind-control-plane kubelet[271]: E0305 10:29:59.420037     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:29:59 kind-control-plane kubelet[271]: E0305 10:29:59.520338     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:29:59 kind-control-plane kubelet[271]: E0305 10:29:59.620567     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:29:59 kind-control-plane kubelet[271]: W0305 10:29:59.688473     271 status_manager.go:530] Failed to get status for pod "etcd-kind-control-plane_kube-system(38b67f836febc1aa47393a3432e1a094)": Get https://172.18.0.2:6443/api/v1/namespaces/kube-system/pods/etcd-kind-control-plane: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:59 kind-control-plane kubelet[271]: E0305 10:29:59.720772     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:29:59 kind-control-plane kubelet[271]: E0305 10:29:59.818013     271 controller.go:135] failed to ensure node lease exists, will retry in 3.2s, error: Get https://172.18.0.2:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/kind-control-plane?timeout=10s: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:59 kind-control-plane kubelet[271]: E0305 10:29:59.821048     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:29:59 kind-control-plane kubelet[271]: E0305 10:29:59.888360     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: Get https://172.18.0.2:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:29:59 kind-control-plane kubelet[271]: E0305 10:29:59.921328     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:00 kind-control-plane kubelet[271]: E0305 10:30:00.021550     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:00 kind-control-plane kubelet[271]: E0305 10:30:00.088537     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://172.18.0.2:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:30:00 kind-control-plane kubelet[271]: E0305 10:30:00.121740     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:00 kind-control-plane kubelet[271]: E0305 10:30:00.221922     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:00 kind-control-plane kubelet[271]: E0305 10:30:00.288407     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: Get https://172.18.0.2:6443/api/v1/nodes?fieldSelector=metadata.name%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:30:00 kind-control-plane kubelet[271]: E0305 10:30:00.322232     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:00 kind-control-plane kubelet[271]: E0305 10:30:00.422452     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:00 kind-control-plane kubelet[271]: E0305 10:30:00.488440     271 kubelet_node_status.go:92] Unable to register node "kind-control-plane" with API server: Post https://172.18.0.2:6443/api/v1/nodes: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:30:00 kind-control-plane kubelet[271]: E0305 10:30:00.522660     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:00 kind-control-plane kubelet[271]: E0305 10:30:00.622832     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:00 kind-control-plane kubelet[271]: E0305 10:30:00.723038     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:00 kind-control-plane kubelet[271]: E0305 10:30:00.823260     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:00 kind-control-plane kubelet[271]: E0305 10:30:00.888527     271 reflector.go:156] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: Get https://172.18.0.2:6443/apis/storage.k8s.io/v1beta1/csidrivers?limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:30:00 kind-control-plane kubelet[271]: E0305 10:30:00.923508     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:01 kind-control-plane kubelet[271]: E0305 10:30:01.023703     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:01 kind-control-plane kubelet[271]: E0305 10:30:01.088406     271 reflector.go:156] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: Get https://172.18.0.2:6443/apis/node.k8s.io/v1beta1/runtimeclasses?limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:30:01 kind-control-plane kubelet[271]: E0305 10:30:01.123977     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:01 kind-control-plane kubelet[271]: E0305 10:30:01.224197     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:01 kind-control-plane kubelet[271]: E0305 10:30:01.324402     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:01 kind-control-plane kubelet[271]: E0305 10:30:01.424614     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:01 kind-control-plane kubelet[271]: E0305 10:30:01.488494     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:449: Failed to list *v1.Service: Get https://172.18.0.2:6443/api/v1/services?limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:30:01 kind-control-plane kubelet[271]: E0305 10:30:01.524822     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:01 kind-control-plane kubelet[271]: E0305 10:30:01.624978     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:01 kind-control-plane kubelet[271]: E0305 10:30:01.688519     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://172.18.0.2:6443/api/v1/pods?fieldSelector=spec.nodeName%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:30:01 kind-control-plane kubelet[271]: E0305 10:30:01.725225     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:01 kind-control-plane kubelet[271]: E0305 10:30:01.825445     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:01 kind-control-plane kubelet[271]: E0305 10:30:01.861406     271 kubelet.go:2183] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized
Mar 05 10:30:01 kind-control-plane kubelet[271]: E0305 10:30:01.888491     271 reflector.go:156] k8s.io/kubernetes/pkg/kubelet/kubelet.go:458: Failed to list *v1.Node: Get https://172.18.0.2:6443/api/v1/nodes?fieldSelector=metadata.name%3Dkind-control-plane&limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:30:01 kind-control-plane kubelet[271]: E0305 10:30:01.925635     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:02 kind-control-plane kubelet[271]: E0305 10:30:02.025822     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:02 kind-control-plane kubelet[271]: E0305 10:30:02.088487     271 csi_plugin.go:267] Failed to initialize CSINodeInfo: error updating CSINode annotation: timed out waiting for the condition; caused by: Get https://172.18.0.2:6443/apis/storage.k8s.io/v1/csinodes/kind-control-plane: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:30:02 kind-control-plane kubelet[271]: I0305 10:30:02.088631     271 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Mar 05 10:30:02 kind-control-plane kubelet[271]: I0305 10:30:02.089562     271 kubelet_node_status.go:70] Attempting to register node kind-control-plane
Mar 05 10:30:02 kind-control-plane kubelet[271]: E0305 10:30:02.125997     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:02 kind-control-plane kubelet[271]: E0305 10:30:02.226200     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:02 kind-control-plane kubelet[271]: E0305 10:30:02.288528     271 reflector.go:156] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.CSIDriver: Get https://172.18.0.2:6443/apis/storage.k8s.io/v1beta1/csidrivers?limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:30:02 kind-control-plane kubelet[271]: E0305 10:30:02.323939     271 event.go:272] Unable to write event: 'Post https://172.18.0.2:6443/api/v1/namespaces/default/events: dial tcp 172.18.0.2:6443: connect: connection refused' (may retry after sleeping)
Mar 05 10:30:02 kind-control-plane kubelet[271]: E0305 10:30:02.326405     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:02 kind-control-plane kubelet[271]: E0305 10:30:02.426607     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:02 kind-control-plane kubelet[271]: E0305 10:30:02.488477     271 reflector.go:156] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.RuntimeClass: Get https://172.18.0.2:6443/apis/node.k8s.io/v1beta1/runtimeclasses?limit=500&resourceVersion=0: dial tcp 172.18.0.2:6443: connect: connection refused
Mar 05 10:30:02 kind-control-plane kubelet[271]: E0305 10:30:02.526867     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:02 kind-control-plane kubelet[271]: E0305 10:30:02.627095     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:02 kind-control-plane kubelet[271]: E0305 10:30:02.727293     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:02 kind-control-plane kubelet[271]: E0305 10:30:02.827449     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:02 kind-control-plane kubelet[271]: E0305 10:30:02.927652     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:03 kind-control-plane kubelet[271]: E0305 10:30:03.027852     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:03 kind-control-plane kubelet[271]: E0305 10:30:03.128023     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:03 kind-control-plane kubelet[271]: I0305 10:30:03.140940     271 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Mar 05 10:30:03 kind-control-plane kubelet[271]: I0305 10:30:03.143157     271 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Mar 05 10:30:03 kind-control-plane kubelet[271]: I0305 10:30:03.144275     271 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Mar 05 10:30:03 kind-control-plane kubelet[271]: I0305 10:30:03.148052     271 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Mar 05 10:30:03 kind-control-plane kubelet[271]: E0305 10:30:03.228216     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:03 kind-control-plane kubelet[271]: E0305 10:30:03.328407     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:03 kind-control-plane kubelet[271]: E0305 10:30:03.428600     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:03 kind-control-plane kubelet[271]: E0305 10:30:03.528802     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:03 kind-control-plane kubelet[271]: E0305 10:30:03.628989     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:03 kind-control-plane kubelet[271]: E0305 10:30:03.729181     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:03 kind-control-plane kubelet[271]: E0305 10:30:03.829398     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:03 kind-control-plane kubelet[271]: E0305 10:30:03.929659     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:04 kind-control-plane kubelet[271]: E0305 10:30:04.029823     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:04 kind-control-plane kubelet[271]: E0305 10:30:04.129936     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:04 kind-control-plane kubelet[271]: I0305 10:30:04.149250     271 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Mar 05 10:30:04 kind-control-plane kubelet[271]: I0305 10:30:04.149282     271 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Mar 05 10:30:04 kind-control-plane kubelet[271]: I0305 10:30:04.149250     271 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Mar 05 10:30:04 kind-control-plane kubelet[271]: I0305 10:30:04.149408     271 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Mar 05 10:30:04 kind-control-plane kubelet[271]: E0305 10:30:04.230034     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:04 kind-control-plane kubelet[271]: E0305 10:30:04.330202     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:04 kind-control-plane kubelet[271]: E0305 10:30:04.430419     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:04 kind-control-plane kubelet[271]: E0305 10:30:04.530639     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:04 kind-control-plane kubelet[271]: E0305 10:30:04.630863     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:04 kind-control-plane kubelet[271]: E0305 10:30:04.731093     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:04 kind-control-plane kubelet[271]: E0305 10:30:04.831341     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:04 kind-control-plane kubelet[271]: E0305 10:30:04.931865     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:05 kind-control-plane kubelet[271]: E0305 10:30:05.032091     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:05 kind-control-plane kubelet[271]: E0305 10:30:05.132287     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:05 kind-control-plane kubelet[271]: E0305 10:30:05.232501     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:05 kind-control-plane kubelet[271]: E0305 10:30:05.332723     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:05 kind-control-plane kubelet[271]: E0305 10:30:05.432887     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:05 kind-control-plane kubelet[271]: E0305 10:30:05.533085     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:05 kind-control-plane kubelet[271]: E0305 10:30:05.633260     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:05 kind-control-plane kubelet[271]: E0305 10:30:05.733469     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:05 kind-control-plane kubelet[271]: E0305 10:30:05.833692     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:05 kind-control-plane kubelet[271]: E0305 10:30:05.933906     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:06 kind-control-plane kubelet[271]: E0305 10:30:06.034130     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:06 kind-control-plane kubelet[271]: E0305 10:30:06.120787     271 controller.go:227] failed to get node "kind-control-plane" when trying to set owner ref to the node lease: nodes "kind-control-plane" not found
Mar 05 10:30:06 kind-control-plane kubelet[271]: E0305 10:30:06.135199     271 kubelet.go:2263] node "kind-control-plane" not found
Mar 05 10:30:06 kind-control-plane kubelet[271]: I0305 10:30:06.200002     271 kubelet_node_status.go:73] Successfully registered node kind-control-plane
Mar 05 10:30:06 kind-control-plane kubelet[271]: I0305 10:30:06.234575     271 reconciler.go:156] Reconciler: start to sync state
Mar 05 10:30:06 kind-control-plane kubelet[271]: E0305 10:30:06.862200     271 kubelet.go:2183] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized
Mar 05 10:30:06 kind-control-plane kubelet[271]: E0305 10:30:06.878594     271 summary_sys_containers.go:47] Failed to get system container stats for "/kubepods": failed to get cgroup stats for "/kubepods": failed to get container info for "/kubepods": unknown container "/kubepods"
Mar 05 10:30:06 kind-control-plane kubelet[271]: E0305 10:30:06.878625     271 helpers.go:680] eviction manager: failed to construct signal: "allocatableMemory.available" error: system container "pods" not found in metrics
Mar 05 10:30:08 kind-control-plane systemd[1]: Stopping kubelet: The Kubernetes Node Agent...
Mar 05 10:30:08 kind-control-plane systemd[1]: kubelet.service: Succeeded.
Mar 05 10:30:08 kind-control-plane systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
Mar 05 10:30:08 kind-control-plane systemd[1]: Started kubelet: The Kubernetes Node Agent.
Mar 05 10:30:08 kind-control-plane kubelet[727]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 05 10:30:08 kind-control-plane kubelet[727]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 05 10:30:08 kind-control-plane kubelet[727]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 05 10:30:08 kind-control-plane kubelet[727]: I0305 10:30:08.825670     727 server.go:416] Version: v1.17.0
Mar 05 10:30:08 kind-control-plane kubelet[727]: I0305 10:30:08.825928     727 plugins.go:100] No cloud provider specified.
Mar 05 10:30:08 kind-control-plane kubelet[727]: I0305 10:30:08.825962     727 server.go:821] Client rotation is on, will bootstrap in background
Mar 05 10:30:08 kind-control-plane kubelet[727]: I0305 10:30:08.827614     727 certificate_store.go:129] Loading cert/key pair from "/var/lib/kubelet/pki/kubelet-client-current.pem".
Mar 05 10:30:08 kind-control-plane kubelet[727]: W0305 10:30:08.828613     727 container_manager_linux.go:843] CPUAccounting not enabled for pid: 727
Mar 05 10:30:08 kind-control-plane kubelet[727]: W0305 10:30:08.828629     727 container_manager_linux.go:846] MemoryAccounting not enabled for pid: 727
Mar 05 10:30:49 kind-control-plane kubelet[727]: I0305 10:30:49.349972     727 server.go:641] --cgroups-per-qos enabled, but --cgroup-root was not specified.  defaulting to /
Mar 05 10:30:49 kind-control-plane kubelet[727]: I0305 10:30:49.350428     727 container_manager_linux.go:265] container manager verified user specified cgroup-root exists: []
Mar 05 10:30:49 kind-control-plane kubelet[727]: I0305 10:30:49.350449     727 container_manager_linux.go:270] Creating Container Manager object based on Node Config: {RuntimeCgroupsName: SystemCgroupsName: KubeletCgroupsName: ContainerRuntime:remote CgroupsPerQOS:true CgroupRoot:/ CgroupDriver:cgroupfs KubeletRootDir:/var/lib/kubelet ProtectKernelDefaults:false NodeAllocatableConfig:{KubeReservedCgroupName: SystemReservedCgroupName: ReservedSystemCPUs: EnforceNodeAllocatable:map[pods:{}] KubeReserved:map[] SystemReserved:map[] HardEvictionThresholds:[]} QOSReserved:map[] ExperimentalCPUManagerPolicy:none ExperimentalCPUManagerReconcilePeriod:10s ExperimentalPodPidsLimit:-1 EnforceCPULimits:true CPUCFSQuotaPeriod:100ms ExperimentalTopologyManagerPolicy:none}
Mar 05 10:30:49 kind-control-plane kubelet[727]: I0305 10:30:49.350585     727 fake_topology_manager.go:29] [fake topologymanager] NewFakeManager
Mar 05 10:30:49 kind-control-plane kubelet[727]: I0305 10:30:49.350604     727 container_manager_linux.go:305] Creating device plugin manager: true
Mar 05 10:30:49 kind-control-plane kubelet[727]: I0305 10:30:49.350637     727 fake_topology_manager.go:39] [fake topologymanager] AddHintProvider HintProvider:  &{kubelet.sock /var/lib/kubelet/device-plugins/ map[] {0 0} <nil> {{} [0 0 0]} 0x1b084c0 0x3f47888 0x1b08d90 map[] map[] map[] map[] map[] 0xc00077f320 [0] 0x3f47888}
Mar 05 10:30:49 kind-control-plane kubelet[727]: I0305 10:30:49.350680     727 state_mem.go:36] [cpumanager] initializing new in-memory state store
Mar 05 10:30:49 kind-control-plane kubelet[727]: I0305 10:30:49.350811     727 state_mem.go:84] [cpumanager] updated default cpuset: ""
Mar 05 10:30:49 kind-control-plane kubelet[727]: I0305 10:30:49.350822     727 state_mem.go:92] [cpumanager] updated cpuset assignments: "map[]"
Mar 05 10:30:49 kind-control-plane kubelet[727]: I0305 10:30:49.350833     727 fake_topology_manager.go:39] [fake topologymanager] AddHintProvider HintProvider:  &{{0 0} 0x3f47888 10000000000 0xc000707e00 <nil> <nil> <nil> <nil> map[] 0x3f47888}
Mar 05 10:30:49 kind-control-plane kubelet[727]: I0305 10:30:49.350948     727 kubelet.go:286] Adding pod path: /etc/kubernetes/manifests
Mar 05 10:30:49 kind-control-plane kubelet[727]: I0305 10:30:49.350982     727 kubelet.go:311] Watching apiserver
Mar 05 10:30:49 kind-control-plane kubelet[727]: W0305 10:30:49.353108     727 util_unix.go:103] Using "/run/containerd/containerd.sock" as endpoint is deprecated, please consider using full url format "unix:///run/containerd/containerd.sock".
Mar 05 10:30:49 kind-control-plane kubelet[727]: I0305 10:30:49.353322     727 remote_runtime.go:59] parsed scheme: ""
Mar 05 10:30:49 kind-control-plane kubelet[727]: I0305 10:30:49.353349     727 remote_runtime.go:59] scheme "" not registered, fallback to default scheme
Mar 05 10:30:49 kind-control-plane kubelet[727]: I0305 10:30:49.353398     727 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{/run/containerd/containerd.sock 0  <nil>}] <nil>}
Mar 05 10:30:49 kind-control-plane kubelet[727]: I0305 10:30:49.353430     727 clientconn.go:577] ClientConn switching balancer to "pick_first"
Mar 05 10:30:49 kind-control-plane kubelet[727]: W0305 10:30:49.353465     727 util_unix.go:103] Using "/run/containerd/containerd.sock" as endpoint is deprecated, please consider using full url format "unix:///run/containerd/containerd.sock".
Mar 05 10:30:49 kind-control-plane kubelet[727]: I0305 10:30:49.353502     727 remote_image.go:50] parsed scheme: ""
Mar 05 10:30:49 kind-control-plane kubelet[727]: I0305 10:30:49.353513     727 remote_image.go:50] scheme "" not registered, fallback to default scheme
Mar 05 10:30:49 kind-control-plane kubelet[727]: I0305 10:30:49.353536     727 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{/run/containerd/containerd.sock 0  <nil>}] <nil>}
Mar 05 10:30:49 kind-control-plane kubelet[727]: I0305 10:30:49.353553     727 clientconn.go:577] ClientConn switching balancer to "pick_first"
Mar 05 10:31:09 kind-control-plane kubelet[727]: E0305 10:31:09.727444     727 aws_credentials.go:77] while getting AWS credentials NoCredentialProviders: no valid providers in chain. Deprecated.
Mar 05 10:31:09 kind-control-plane kubelet[727]:         For verbose messaging see aws.Config.CredentialsChainVerboseErrors
Mar 05 10:31:09 kind-control-plane kubelet[727]: I0305 10:31:09.728140     727 kuberuntime_manager.go:211] Container runtime containerd initialized, version: v1.3.2, apiVersion: v1alpha2
Mar 05 10:31:09 kind-control-plane kubelet[727]: I0305 10:31:09.728574     727 server.go:1113] Started kubelet
Mar 05 10:31:09 kind-control-plane kubelet[727]: I0305 10:31:09.728766     727 server.go:143] Starting to listen on 0.0.0.0:10250
Mar 05 10:31:09 kind-control-plane kubelet[727]: I0305 10:31:09.731397     727 server.go:354] Adding debug handlers to kubelet server.
Mar 05 10:31:09 kind-control-plane kubelet[727]: I0305 10:31:09.732557     727 fs_resource_analyzer.go:64] Starting FS ResourceAnalyzer
Mar 05 10:31:09 kind-control-plane kubelet[727]: I0305 10:31:09.733542     727 volume_manager.go:265] Starting Kubelet Volume Manager
Mar 05 10:31:09 kind-control-plane kubelet[727]: I0305 10:31:09.733554     727 desired_state_of_world_populator.go:138] Desired state populator starts to run
Mar 05 10:31:09 kind-control-plane kubelet[727]: E0305 10:31:09.735271     727 kubelet.go:2183] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized
Mar 05 10:31:09 kind-control-plane kubelet[727]: I0305 10:31:09.735424     727 clientconn.go:104] parsed scheme: "unix"
Mar 05 10:31:09 kind-control-plane kubelet[727]: I0305 10:31:09.735445     727 clientconn.go:104] scheme "unix" not registered, fallback to default scheme
Mar 05 10:31:09 kind-control-plane kubelet[727]: I0305 10:31:09.735525     727 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock 0  <nil>}] <nil>}
Mar 05 10:31:09 kind-control-plane kubelet[727]: I0305 10:31:09.735539     727 clientconn.go:577] ClientConn switching balancer to "pick_first"
Mar 05 10:31:09 kind-control-plane kubelet[727]: I0305 10:31:09.748654     727 status_manager.go:157] Starting to sync pod status with apiserver
Mar 05 10:31:09 kind-control-plane kubelet[727]: I0305 10:31:09.748692     727 kubelet.go:1820] Starting kubelet main sync loop.
Mar 05 10:31:09 kind-control-plane kubelet[727]: E0305 10:31:09.748744     727 kubelet.go:1844] skipping pod synchronization - [container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]
Mar 05 10:31:09 kind-control-plane kubelet[727]: I0305 10:31:09.833454     727 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Mar 05 10:31:09 kind-control-plane kubelet[727]: I0305 10:31:09.833488     727 kuberuntime_manager.go:981] updating runtime config through cri with podcidr 10.244.0.0/24
Mar 05 10:31:09 kind-control-plane kubelet[727]: I0305 10:31:09.834288     727 kubelet_network.go:77] Setting Pod CIDR:  -> 10.244.0.0/24
Mar 05 10:31:09 kind-control-plane kubelet[727]: I0305 10:31:09.834463     727 kubelet_node_status.go:70] Attempting to register node kind-control-plane
Mar 05 10:31:09 kind-control-plane kubelet[727]: I0305 10:31:09.839973     727 cpu_manager.go:173] [cpumanager] starting with none policy
Mar 05 10:31:09 kind-control-plane kubelet[727]: I0305 10:31:09.840326     727 cpu_manager.go:174] [cpumanager] reconciling every 10s
Mar 05 10:31:09 kind-control-plane kubelet[727]: I0305 10:31:09.840691     727 policy_none.go:43] [cpumanager] none policy: Start
Mar 05 10:31:09 kind-control-plane kubelet[727]: I0305 10:31:09.840303     727 kubelet_node_status.go:112] Node kind-control-plane was previously registered
Mar 05 10:31:09 kind-control-plane kubelet[727]: F0305 10:31:09.841684     727 kubelet.go:1380] Failed to start ContainerManager failed to initialize top level QOS containers: failed to update top level Burstable QOS cgroup : failed to set supported cgroup subsystems for cgroup [kubepods burstable]: failed to find subsystem mount for required subsystem: pids
Mar 05 10:31:09 kind-control-plane systemd[1]: kubelet.service: Main process exited, code=exited, status=255/EXCEPTION
Mar 05 10:31:09 kind-control-plane systemd[1]: kubelet.service: Failed with result 'exit-code'.
Mar 05 10:31:11 kind-control-plane systemd[1]: kubelet.service: Service RestartSec=1s expired, scheduling restart.
Mar 05 10:31:11 kind-control-plane systemd[1]: kubelet.service: Scheduled restart job, restart counter is at 1.
Mar 05 10:31:11 kind-control-plane systemd[1]: Stopped kubelet: The Kubernetes Node Agent.
Mar 05 10:31:11 kind-control-plane systemd[1]: Started kubelet: The Kubernetes Node Agent.
Mar 05 10:31:11 kind-control-plane kubelet[7109]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 05 10:31:11 kind-control-plane kubelet[7109]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 05 10:31:11 kind-control-plane kubelet[7109]: Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Mar 05 10:31:11 kind-control-plane kubelet[7109]: I0305 10:31:11.133784    7109 server.go:416] Version: v1.17.0
Mar 05 10:31:11 kind-control-plane kubelet[7109]: I0305 10:31:11.134006    7109 plugins.go:100] No cloud provider specified.
Mar 05 10:31:11 kind-control-plane kubelet[7109]: I0305 10:31:11.134024    7109 server.go:821] Client rotation is on, will bootstrap in background
Mar 05 10:31:11 kind-control-plane kubelet[7109]: I0305 10:31:11.135818    7109 certificate_store.go:129] Loading cert/key pair from "/var/lib/kubelet/pki/kubelet-client-current.pem".
Mar 05 10:31:11 kind-control-plane kubelet[7109]: W0305 10:31:11.136675    7109 container_manager_linux.go:843] CPUAccounting not enabled for pid: 7109
Mar 05 10:31:11 kind-control-plane kubelet[7109]: W0305 10:31:11.136696    7109 container_manager_linux.go:846] MemoryAccounting not enabled for pid: 7109
Mar 05 10:31:51 kind-control-plane kubelet[7109]: I0305 10:31:51.747923    7109 server.go:641] --cgroups-per-qos enabled, but --cgroup-root was not specified.  defaulting to /
Mar 05 10:31:51 kind-control-plane kubelet[7109]: I0305 10:31:51.748333    7109 container_manager_linux.go:265] container manager verified user specified cgroup-root exists: []
Mar 05 10:31:51 kind-control-plane kubelet[7109]: I0305 10:31:51.748351    7109 container_manager_linux.go:270] Creating Container Manager object based on Node Config: {RuntimeCgroupsName: SystemCgroupsName: KubeletCgroupsName: ContainerRuntime:remote CgroupsPerQOS:true CgroupRoot:/ CgroupDriver:cgroupfs KubeletRootDir:/var/lib/kubelet ProtectKernelDefaults:false NodeAllocatableConfig:{KubeReservedCgroupName: SystemReservedCgroupName: ReservedSystemCPUs: EnforceNodeAllocatable:map[pods:{}] KubeReserved:map[] SystemReserved:map[] HardEvictionThresholds:[]} QOSReserved:map[] ExperimentalCPUManagerPolicy:none ExperimentalCPUManagerReconcilePeriod:10s ExperimentalPodPidsLimit:-1 EnforceCPULimits:true CPUCFSQuotaPeriod:100ms ExperimentalTopologyManagerPolicy:none}
Mar 05 10:31:51 kind-control-plane kubelet[7109]: I0305 10:31:51.748608    7109 fake_topology_manager.go:29] [fake topologymanager] NewFakeManager
Mar 05 10:31:51 kind-control-plane kubelet[7109]: I0305 10:31:51.748626    7109 container_manager_linux.go:305] Creating device plugin manager: true
Mar 05 10:31:51 kind-control-plane kubelet[7109]: I0305 10:31:51.748670    7109 fake_topology_manager.go:39] [fake topologymanager] AddHintProvider HintProvider:  &{kubelet.sock /var/lib/kubelet/device-plugins/ map[] {0 0} <nil> {{} [0 0 0]} 0x1b084c0 0x3f47888 0x1b08d90 map[] map[] map[] map[] map[] 0xc000843110 [0] 0x3f47888}
Mar 05 10:31:51 kind-control-plane kubelet[7109]: I0305 10:31:51.748712    7109 state_mem.go:36] [cpumanager] initializing new in-memory state store
Mar 05 10:31:51 kind-control-plane kubelet[7109]: I0305 10:31:51.748848    7109 state_mem.go:84] [cpumanager] updated default cpuset: ""
Mar 05 10:31:51 kind-control-plane kubelet[7109]: I0305 10:31:51.748858    7109 state_mem.go:92] [cpumanager] updated cpuset assignments: "map[]"
Mar 05 10:31:51 kind-control-plane kubelet[7109]: I0305 10:31:51.748870    7109 fake_topology_manager.go:39] [fake topologymanager] AddHintProvider HintProvider:  &{{0 0} 0x3f47888 10000000000 0xc0007a7620 <nil> <nil> <nil> <nil> map[] 0x3f47888}
Mar 05 10:31:51 kind-control-plane kubelet[7109]: I0305 10:31:51.749001    7109 kubelet.go:286] Adding pod path: /etc/kubernetes/manifests
Mar 05 10:31:51 kind-control-plane kubelet[7109]: I0305 10:31:51.749034    7109 kubelet.go:311] Watching apiserver
Mar 05 10:31:51 kind-control-plane kubelet[7109]: W0305 10:31:51.751107    7109 util_unix.go:103] Using "/run/containerd/containerd.sock" as endpoint is deprecated, please consider using full url format "unix:///run/containerd/containerd.sock".
Mar 05 10:31:51 kind-control-plane kubelet[7109]: I0305 10:31:51.751221    7109 remote_runtime.go:59] parsed scheme: ""
Mar 05 10:31:51 kind-control-plane kubelet[7109]: I0305 10:31:51.751243    7109 remote_runtime.go:59] scheme "" not registered, fallback to default scheme
Mar 05 10:31:51 kind-control-plane kubelet[7109]: I0305 10:31:51.751273    7109 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{/run/containerd/containerd.sock 0  <nil>}] <nil>}
Mar 05 10:31:51 kind-control-plane kubelet[7109]: I0305 10:31:51.751285    7109 clientconn.go:577] ClientConn switching balancer to "pick_first"
Mar 05 10:31:51 kind-control-plane kubelet[7109]: W0305 10:31:51.751319    7109 util_unix.go:103] Using "/run/containerd/containerd.sock" as endpoint is deprecated, please consider using full url format "unix:///run/containerd/containerd.sock".
Mar 05 10:31:51 kind-control-plane kubelet[7109]: I0305 10:31:51.751348    7109 remote_image.go:50] parsed scheme: ""
Mar 05 10:31:51 kind-control-plane kubelet[7109]: I0305 10:31:51.751357    7109 remote_image.go:50] scheme "" not registered, fallback to default scheme
Mar 05 10:31:51 kind-control-plane kubelet[7109]: I0305 10:31:51.751377    7109 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{/run/containerd/containerd.sock 0  <nil>}] <nil>}
Mar 05 10:31:51 kind-control-plane kubelet[7109]: I0305 10:31:51.751385    7109 clientconn.go:577] ClientConn switching balancer to "pick_first"

I'm not exactly sure what I'm looking for in these logs - if it would be useful to see other logs produced from kind export logs let me known which and I'll share them

MVJosh on 5 Mar 2020

aws/codebuild/standard:2.0 is not a valid image reference, presumably they have some magic for this and you have to build the image too :/

BenTheElder on 5 Mar 2020

It seems building that particular image remotely over ssh hangs on + apt-add-repository -y ppa:git-core/ppa ... so I gave up and started building it on my work macbook instead, which is still running ...

In the meantime I can't seem to find any docs about how codebuild actually runs the image so it's a bit hard to correctly replicate this accurately.

Is it mounting the host docker socket? Is it using docker in docker? What other mounts exist? What does the network look like?

It would be really helpful to have a more specific set of reproducing steps that don't require an account.

FWIW We have KIND working in a lot of other environments at https://github.com/kind-ci/examples including some similar ones (Google Cloud Build, Azure Pipelines), though the contents there are mostly user contributed. I'd love to fix this everywhere but it's a little out of scope for us to focus on here.

... Especially when I have to choose between working on fixing something like #148 and waiting for php to compile in this codebuild image 😞

EDIT: I've killed the build for now will revisit at some other time. I gave it something like 30 minutes but I need my cores back and I'm running out of disk space :+)

BenTheElder on 6 Mar 2020

Thanks for investigating - I don't know enough about CodeBuild to answer those questions unfortunately and the docs aren't too forthcoming about what goes on behind-the-scenes.

I'll look into building the CodeBuild image myself and reproducing locally, and will post back here once I've got some steps to reproduce locally without requiring an AWS account.

MVJosh on 6 Mar 2020

❤2

I am seeing this in another setting, I'm not actually using kind. I am running kublete and containerd in a kata container and using systemd as well.

errordeveloper on 7 Apr 2020

What I'm seeing is this:

root@test-cluster-master-65bcbb95bb-m7fqr:/# ls /sys/fs/cgroup/memory/kubepods.slice
ls: cannot access '/sys/fs/cgroup/memory/kubepods.slice': No such file or directory
root@test-cluster-master-65bcbb95bb-m7fqr:/# ls /sys/fs/cgroup/memory/kubepods      
besteffort             memory.failcnt              memory.kmem.max_usage_in_bytes  memory.kmem.tcp.max_usage_in_bytes  memory.max_usage_in_bytes        memory.memsw.usage_in_bytes      memory.soft_limit_in_bytes  memory.use_hierarchy
cgroup.clone_children  memory.force_empty          memory.kmem.slabinfo            memory.kmem.tcp.usage_in_bytes      memory.memsw.failcnt             memory.move_charge_at_immigrate  memory.stat                 notify_on_release
cgroup.event_control   memory.kmem.failcnt         memory.kmem.tcp.failcnt         memory.kmem.usage_in_bytes          memory.memsw.limit_in_bytes      memory.oom_control               memory.swappiness           tasks
cgroup.procs           memory.kmem.limit_in_bytes  memory.kmem.tcp.limit_in_bytes  memory.limit_in_bytes               memory.memsw.max_usage_in_bytes  memory.pressure_level            memory.usage_in_bytes
root@test-cluster-master-65bcbb95bb-m7fqr:/# systemctl cat kubepods.slice
# /run/systemd/transient/kubepods.slice
# This is a transient unit file, created programmatically via the systemd API. Do not edit.
[Unit]
Description=libcontainer container kubepods.slice
Wants=-.slice

[Slice]
MemoryAccounting=yes
CPUAccounting=yes
BlockIOAccounting=yes

[Unit]
DefaultDependencies=no

[Slice]
MemoryLimit=4180226048
CPUShares=2048
TasksAccounting=yes
TasksMax=73728
root@test-cluster-master-65bcbb95bb-m7fqr:/#

I think the issue is basically that there is no kubepods.slice in sysfs, it's just kubepods. In my case systemd is actually running in that cgroup.

errordeveloper on 7 Apr 2020

lacking evidence of a change to kind or any activity I'm going to ask that we move discussion for this CI platform to https://github.com/kind-ci/examples where we have many working CI with community support rather than this tracker.

BenTheElder on 30 Apr 2020

Was this page helpful?

0 / 5 - 0 ratings