K3d: default-scheduler 0/2 nodes are available: 2 node(s) had taints that the pod didn't tolerate.

[debianmaster]

brand new cluster with v1.0.0 . failed with $subject error

bash-3.2$ k3d create -n clstr1 --workers=1
2019/04/28 17:32:31 Created cluster network with ID e7d66d1ce94276e4bf596cb947288b898b0273608bcab25f014d81c33ebf4bad
2019/04/28 17:32:31 Creating cluster [clstr1]
2019/04/28 17:32:31 Creating server using docker.io/rancher/k3s:v0.4.0...
2019/04/28 17:32:36 Booting 1 workers for cluster clstr1
Created worker with ID cf91e20377cc3832488468e632cb3c64986a5dba080150a5904c9a3d3589ac9a
2019/04/28 17:32:41 SUCCESS: created cluster [clstr1]
2019/04/28 17:32:41 You can now use the cluster with:

export KUBECONFIG="$(k3d get-kubeconfig --name='clstr1')"
kubectl cluster-info
bash-3.2$ export KUBECONFIG="$(k3d get-kubeconfig --name='clstr1')"
bash-3.2$ kubectl cluster-info
Kubernetes master is running at https://localhost:6443
CoreDNS is running at https://localhost:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
bash-3.2$ k get nodes
NAME           STATUS    ROLES     AGE       VERSION
cf91e20377cc   Ready     <none>    8s        v1.14.1-k3s.4
d364f0d492c5   Ready     <none>    8s        v1.14.1-k3s.4
bash-3.2$ k get po --all-namespaces
NAMESPACE     NAME                         READY     STATUS    RESTARTS   AGE
kube-system   coredns-857cdbd8b4-6kc6c     0/1       Pending   0          8s
kube-system   helm-install-traefik-bmdg7   0/1       Pending   0          8s
bash-3.2$ k get po --all-namespaces
NAMESPACE     NAME                         READY     STATUS    RESTARTS   AGE
kube-system   coredns-857cdbd8b4-6kc6c     0/1       Pending   0          9s
kube-system   helm-install-traefik-bmdg7   0/1       Pending   0          9s

k describe po coredns-857cdbd8b4-6kc6c -n kube-system
Name:               coredns-857cdbd8b4-6kc6c
Namespace:          kube-system
Priority:           0
PriorityClassName:  <none>
Node:               <none>
Labels:             k8s-app=kube-dns
                    pod-template-hash=857cdbd8b4
Annotations:        <none>
Status:             Pending
IP:
Controlled By:      ReplicaSet/coredns-857cdbd8b4
Containers:
  coredns:
    Image:       coredns/coredns:1.3.0
    Ports:       53/UDP, 53/TCP, 9153/TCP
    Host Ports:  0/UDP, 0/TCP, 0/TCP
    Args:
      -conf
      /etc/coredns/Corefile
    Limits:
      memory:  170Mi
    Requests:
      cpu:        100m
      memory:     70Mi
    Liveness:     http-get http://:8080/health delay=60s timeout=5s period=10s #success=1 #failure=5
    Environment:  <none>
    Mounts:
      /etc/coredns from config-volume (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from coredns-token-nzxcf (ro)
Conditions:
  Type           Status
  PodScheduled   False
Volumes:
  config-volume:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      coredns
    Optional:  false
  coredns-token-nzxcf:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  coredns-token-nzxcf
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  beta.kubernetes.io/os=linux
Tolerations:     CriticalAddonsOnly
                 node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason            Age              From               Message
  ----     ------            ----             ----               -------
  Warning  FailedScheduling  8s (x4 over 1m)  default-scheduler  0/2 nodes are available: 2 node(s) had taints that the pod didn't tolerate.

[debianmaster]

Name:               6c2ec9d3c5e7
Roles:              <none>
Labels:             beta.kubernetes.io/arch=amd64
                    beta.kubernetes.io/os=linux
                    kubernetes.io/arch=amd64
                    kubernetes.io/hostname=6c2ec9d3c5e7
                    kubernetes.io/os=linux
Annotations:        flannel.alpha.coreos.com/backend-data={"VtepMAC":"aa:7e:e2:05:24:59"}
                    flannel.alpha.coreos.com/backend-type=vxlan
                    flannel.alpha.coreos.com/kube-subnet-manager=true
                    flannel.alpha.coreos.com/public-ip=172.26.0.2
                    node.alpha.kubernetes.io/ttl=0
                    volumes.kubernetes.io/controller-managed-attach-detach=true
CreationTimestamp:  Sun, 28 Apr 2019 17:48:04 +0800
Taints:             node.kubernetes.io/disk-pressure:NoSchedule
Unschedulable:      false
Conditions:
  Type             Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message
  ----             ------  -----------------                 ------------------                ------                       -------
  MemoryPressure   False   Sun, 28 Apr 2019 17:49:14 +0800   Sun, 28 Apr 2019 17:48:04 +0800   KubeletHasSufficientMemory   kubelet has sufficient memory available
  DiskPressure     True    Sun, 28 Apr 2019 17:49:14 +0800   Sun, 28 Apr 2019 17:48:14 +0800   KubeletHasDiskPressure       kubelet has disk pressure
  PIDPressure      False   Sun, 28 Apr 2019 17:49:14 +0800   Sun, 28 Apr 2019 17:48:04 +0800   KubeletHasSufficientPID      kubelet has sufficient PID available
  Ready            True    Sun, 28 Apr 2019 17:49:14 +0800   Sun, 28 Apr 2019 17:48:04 +0800   KubeletReady                 kubelet is posting ready status
Addresses:
  InternalIP:  172.26.0.2
  Hostname:    6c2ec9d3c5e7
Capacity:
 cpu:                6
 ephemeral-storage:  61255492Ki
 hugepages-1Gi:      0
 hugepages-2Mi:      0
 memory:             8164444Ki
 pods:               110
Allocatable:
 cpu:                6
 ephemeral-storage:  59589342571
 hugepages-1Gi:      0
 hugepages-2Mi:      0
 memory:             8164444Ki
 pods:               110
System Info:
 Machine ID:                 
 System UUID:                2FCA4403-0000-0000-A2B2-BD75FC49ED82
 Boot ID:                    b765e50e-c06e-4873-9b27-213a13d5bb2c
 Kernel Version:             4.9.125-linuxkit
 OS Image:                   Unknown
 Operating System:           linux
 Architecture:               amd64
 Container Runtime Version:  containerd://1.2.5+unknown
 Kubelet Version:            v1.14.1-k3s.4
 Kube-Proxy Version:         v1.14.1-k3s.4
PodCIDR:                     10.42.0.0/24
Non-terminated Pods:         (0 in total)
  Namespace                  Name    CPU Requests  CPU Limits  Memory Requests  Memory Limits
  ---------                  ----    ------------  ----------  ---------------  -------------
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  Resource  Requests  Limits
  --------  --------  ------
  cpu       0 (0%)    0 (0%)
  memory    0 (0%)    0 (0%)
Events:
  Type     Reason                   Age               From                   Message
  ----     ------                   ----              ----                   -------
  Normal   Starting                 1m                kubelet, 6c2ec9d3c5e7  Starting kubelet.
  Warning  InvalidDiskCapacity      1m                kubelet, 6c2ec9d3c5e7  invalid capacity 0 on image filesystem
  Normal   NodeHasSufficientMemory  1m (x2 over 1m)   kubelet, 6c2ec9d3c5e7  Node 6c2ec9d3c5e7 status is now: NodeHasSufficientMemory
  Normal   NodeHasNoDiskPressure    1m (x2 over 1m)   kubelet, 6c2ec9d3c5e7  Node 6c2ec9d3c5e7 status is now: NodeHasNoDiskPressure
  Normal   NodeHasSufficientPID     1m (x2 over 1m)   kubelet, 6c2ec9d3c5e7  Node 6c2ec9d3c5e7 status is now: NodeHasSufficientPID
  Normal   NodeAllocatableEnforced  1m                kubelet, 6c2ec9d3c5e7  Updated Node Allocatable limit across pods
  Normal   NodeHasDiskPressure      1m                kubelet, 6c2ec9d3c5e7  Node 6c2ec9d3c5e7 status is now: NodeHasDiskPressure
  Warning  EvictionThresholdMet     8s (x10 over 1m)  kubelet, 6c2ec9d3c5e7  Attempting to reclaim ephemeral-storage

[debianmaster]

issue with my docker. closing

[kkimdev]

@debianmaster What was the issue? I'm hitting the same "FailedScheduling" problem.

[olup]

@debianmaster Having same issue and same logs, can you advise on what was your solution ? Cheers.

[iwilltry42]

Just in general, a rude docker system prune -a and docker network prune can clean up an existing mess in your docker. But only run them if you can loose stuff that's currently not being used (dangling images, stopped containers, etc.).
Other than that, the DOCKER_* environment variables might be causing issues.

Info about the environment, OS and k3d version can be helpful for debugging :+1:

[olup]

Thanks for your quick answer @iwilltry42 . This indeed did the trick.

[debianmaster]

I had restarted my docker engine and it worked. not sure of the root cause. @olup @kkimdev