Flannel: kube-flannel: operation not supported

[tomdee]

Using the latest master build of flannel, I'm seeing the following error when I try to run flanneld with --kube-subnet-mgr

2016-09-21T23:09:04Z E0921 23:09:04.150369       6 network.go:106] failed to register network: operation not supported

I'm not sure what I'm missing to get this running

[mlbiam]

@luxas i'm getting this same error on arm with the flannel yaml. I tried bumping to quay.io/coreos/flannel:v0.6.2-arm but it doesn't like -kube-subnet-mgr

[mlbiam]

I tried rebuilding my cluster to make sure it was fresh. Here's the logs I get from the flannel docker image with debugging enabled:

I1007 10:32:30.975393       1 round_trippers.go:299] curl -k -v -XGET  -H "User-Agent: flanneld/v1.4.0 (linux/arm) kubernetes/$Format" -H "Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkZWZhdWx0LXRva2VuLTM4MmgyIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImRlZmF1bHQiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiIxZmNiYzgwNC04Yzc4LTExZTYtOTBiOS1iODI3ZWI2NDUzNzQiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZS1zeXN0ZW06ZGVmYXVsdCJ9.PpEau6WesQfvLhQ560ioXsiYqmtE-oH9smCbMeX8kO9bi2M_nKlUxSmgbLCk1-jh0FhrpK1n_qDSTqdk1UGmNM8DB1CdJwRH1QaiVmwo9k0yn4tU-I10MB7jQHzLJAlKKCoFR0nwzxPTTdRdcHvW3sz_0h8GhLOF4tdWOv70qO_UfqGjDCQrHR04RScn-pa4R1tUiisnPpFkI3H0l5k2qJHpoWC4xzGA83H-flLzScHpIIdCN0o6armiSNplF17LOPvK6XW5Z6PSaZBGnI5-zb3GJMK7JIJHArE7Bs_EZqCoz-kKryUIpVeL8i84Y0XVcGK3rg-4ogaHPYYx1d70EA" -H "Accept: application/json, */*" https://100.64.0.1:443/api/v1/namespaces/kube-system/pods/kube-flannel-ds-2rs5n
I1007 10:32:31.157899       1 round_trippers.go:318] GET https://100.64.0.1:443/api/v1/namespaces/kube-system/pods/kube-flannel-ds-2rs5n 200 OK in 182 milliseconds
I1007 10:32:31.158467       1 round_trippers.go:324] Response Headers:
I1007 10:32:31.158591       1 round_trippers.go:327]     Content-Type: application/json
I1007 10:32:31.158682       1 round_trippers.go:327]     Date: Fri, 07 Oct 2016 10:32:31 GMT
I1007 10:32:31.159978       1 request.go:908] Response Body: {"kind":"Pod","apiVersion":"v1","metadata":{"name":"kube-flannel-ds-2rs5n","generateName":"kube-flannel-ds-","namespace":"kube-system","selfLink":"/api/v1/namespaces/kube-system/pods/kube-flannel-ds-2rs5n","uid":"5913a5b6-8c79-11e6-90b9-b827eb645374","resourceVersion":"1882","creationTimestamp":"2016-10-07T10:32:29Z","labels":{"app":"flannel","tier":"node"},"annotations":{"kubernetes.io/created-by":"{\"kind\":\"SerializedReference\",\"apiVersion\":\"v1\",\"reference\":{\"kind\":\"DaemonSet\",\"namespace\":\"kube-system\",\"name\":\"kube-flannel-ds\",\"uid\":\"590d2ce9-8c79-11e6-90b9-b827eb645374\",\"apiVersion\":\"extensions\",\"resourceVersion\":\"1878\"}}\n"}},"spec":{"volumes":[{"name":"run","hostPath":{"path":"/run"}},{"name":"cni","hostPath":{"path":"/etc/cni/net.d"}},{"name":"flannel-cfg","configMap":{"name":"kube-flannel-cfg","defaultMode":420}},{"name":"default-token-382h2","secret":{"secretName":"default-token-382h2","defaultMode":420}}],"containers":[{"name":"kube-flannel","image":"quay.io/coreos/flannel-git:v0.6.1-28-g5dde68d-arm","command":["/opt/bin/flanneld","--ip-masq","--kube-subnet-mgr","--v=11"],"env":[{"name":"POD_NAME","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"metadata.name"}}},{"name":"POD_NAMESPACE","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"metadata.namespace"}}}],"resources":{},"volumeMounts":[{"name":"run","mountPath":"/run"},{"name":"flannel-cfg","mountPath":"/etc/kube-flannel/"},{"name":"default-token-382h2","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","imagePullPolicy":"IfNotPresent","securityContext":{"privileged":true}},{"name":"install-cni","image":"quay.io/coreos/flannel-git:v0.6.1-28-g5dde68d-arm","command":["/bin/sh","-c","set -e -x; cp -f /etc/kube-flannel/cni-conf.json /etc/cni/net.d/10-flannel.conf; while true; do sleep 3600; done"],"resources":{},"volumeMounts":[{"name":"cni","mountPath":"/etc/cni/net.d"},{"name":"flannel-cfg","mountPath":"/etc/kube-flannel/"},{"name":"default-token-382h2","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","nodeSelector":{"beta.kubernetes.io/arch":"arm"},"serviceAccountName":"default","serviceAccount":"default","nodeName":"k8s-pi-master","hostNetwork":true,"securityContext":{}},"status":{"phase":"Pending","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2016-10-07T10:32:29Z"},{"type":"Ready","status":"False","lastProbeTime":null,"lastTransitionTime":"2016-10-07T10:32:29Z","reason":"ContainersNotReady","message":"containers with unready status: [kube-flannel install-cni]"}],"hostIP":"192.168.3.50","podIP":"192.168.3.50","startTime":"2016-10-07T10:32:29Z","containerStatuses":[{"name":"install-cni","state":{"waiting":{"reason":"ContainerCreating"}},"lastState":{},"ready":false,"restartCount":0,"image":"quay.io/coreos/flannel-git:v0.6.1-28-g5dde68d-arm","imageID":""},{"name":"kube-flannel","state":{"waiting":{"reason":"ContainerCreating"}},"lastState":{},"ready":false,"restartCount":0,"image":"quay.io/coreos/flannel-git:v0.6.1-28-g5dde68d-arm","imageID":""}]}}
I1007 10:32:31.164071       1 main.go:132] Installing signal handlers
I1007 10:32:31.164442       1 kube.go:233] starting kube subnet manager
I1007 10:32:31.164773       1 reflector.go:211] Starting reflector *api.Node (5m0s) from github.com/coreos/flannel/subnet/kube/kube.go:234
I1007 10:32:31.164883       1 manager.go:133] Determining IP address of default interface
I1007 10:32:31.164889       1 reflector.go:249] Listing and watching *api.Node from github.com/coreos/flannel/subnet/kube/kube.go:234
I1007 10:32:31.166174       1 manager.go:163] Using 192.168.3.50 as external interface
I1007 10:32:31.166266       1 round_trippers.go:299] curl -k -v -XGET  -H "User-Agent: flanneld/v1.4.0 (linux/arm) kubernetes/$Format" -H "Accept: application/json, */*" -H "Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkZWZhdWx0LXRva2VuLTM4MmgyIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImRlZmF1bHQiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiIxZmNiYzgwNC04Yzc4LTExZTYtOTBiOS1iODI3ZWI2NDUzNzQiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZS1zeXN0ZW06ZGVmYXVsdCJ9.PpEau6WesQfvLhQ560ioXsiYqmtE-oH9smCbMeX8kO9bi2M_nKlUxSmgbLCk1-jh0FhrpK1n_qDSTqdk1UGmNM8DB1CdJwRH1QaiVmwo9k0yn4tU-I10MB7jQHzLJAlKKCoFR0nwzxPTTdRdcHvW3sz_0h8GhLOF4tdWOv70qO_UfqGjDCQrHR04RScn-pa4R1tUiisnPpFkI3H0l5k2qJHpoWC4xzGA83H-flLzScHpIIdCN0o6armiSNplF17LOPvK6XW5Z6PSaZBGnI5-zb3GJMK7JIJHArE7Bs_EZqCoz-kKryUIpVeL8i84Y0XVcGK3rg-4ogaHPYYx1d70EA" https://100.64.0.1:443/api/v1/nodes?resourceVersion=0
I1007 10:32:31.166274       1 manager.go:164] Using 192.168.3.50 as external endpoint
E1007 10:32:31.170991       1 network.go:106] failed to register network: operation not supported
I1007 10:32:31.179923       1 round_trippers.go:318] GET https://100.64.0.1:443/api/v1/nodes?resourceVersion=0 200 OK in 13 milliseconds
I1007 10:32:31.180030       1 round_trippers.go:324] Response Headers:
I1007 10:32:31.180074       1 round_trippers.go:327]     Content-Type: application/json
I1007 10:32:31.180111       1 round_trippers.go:327]     Date: Fri, 07 Oct 2016 10:32:31 GMT
I1007 10:32:31.180603       1 request.go:908] Response Body: {"kind":"NodeList","apiVersion":"v1","metadata":{"selfLink":"/api/v1/nodes","resourceVersion":"1873"},"items":[{"metadata":{"name":"k8s-pi-master","selfLink":"/api/v1/nodes/k8s-pi-master","uid":"216836f7-8c78-11e6-90b9-b827eb645374","resourceVersion":"1873","creationTimestamp":"2016-10-07T10:23:46Z","labels":{"beta.kubernetes.io/arch":"arm","beta.kubernetes.io/os":"linux","kubeadm.alpha.kubernetes.io/role":"master","kubernetes.io/hostname":"k8s-pi-master"},"annotations":{"scheduler.alpha.kubernetes.io/taints":"[{\"key\":\"dedicated\",\"value\":\"master\",\"effect\":\"NoSchedule\"}]","volumes.kubernetes.io/controller-managed-attach-detach":"true"}},"spec":{"externalID":"k8s-pi-master"},"status":{"capacity":{"alpha.kubernetes.io/nvidia-gpu":"0","cpu":"4","memory":"948012Ki","pods":"110"},"allocatable":{"alpha.kubernetes.io/nvidia-gpu":"0","cpu":"4","memory":"948012Ki","pods":"110"},"conditions":[{"type":"OutOfDisk","status":"False","lastHeartbeatTime":"2016-10-07T10:32:27Z","lastTransitionTime":"2016-10-07T10:23:46Z","reason":"KubeletHasSufficientDisk","message":"kubelet has sufficient disk space available"},{"type":"MemoryPressure","status":"False","lastHeartbeatTime":"2016-10-07T10:32:27Z","lastTransitionTime":"2016-10-07T10:23:46Z","reason":"KubeletHasSufficientMemory","message":"kubelet has sufficient memory available"},{"type":"DiskPressure","status":"False","lastHeartbeatTime":"2016-10-07T10:32:27Z","lastTransitionTime":"2016-10-07T10:23:46Z","reason":"KubeletHasNoDiskPressure","message":"kubelet has no disk pressure"},{"type":"Ready","status":"True","lastHeartbeatTime":"2016-10-07T10:32:27Z","lastTransitionTime":"2016-10-07T10:23:46Z","reason":"KubeletReady","message":"kubelet is posting ready status. WARNING: CPU hardcapping unsupported"}],"addresses":[{"type":"LegacyHostIP","address":"192.168.3.50"},{"type":"InternalIP","address":"192.168.3.50"}],"daemonEndpoints":{"kubeletEndpoint":{"Port":10250}},"nodeInfo":{"machineID":"f010896c4d7443088901ba263eceffee","systemUUID":"f010896c4d7443088901ba263eceffee","bootID":"112ec821-98e2-4938-a6d0-cfdb2b04ceb7","kernelVersion":"4.1.19-v7+","osImage":"Ubuntu 16.04 LTS","containerRuntimeVersion":"docker://1.11.2","kubeletVersion":"v1.4.0","kubeProxyVersion":"v1.4.0","operatingSystem":"linux","architecture":"arm"},"images":[{"names":["gcr.io/google_containers/kube-proxy-arm:v1.4.0"],"sizeBytes":177088263},{"names":["gcr.io/google_containers/kube-discovery-arm:1.0"],"sizeBytes":122116943},{"names":["gcr.io/google_containers/kube-apiserver-arm:v1.4.0"],"sizeBytes":116704924},{"names":["\u003cnone\u003e:\u003cnone\u003e","\u003cnone\u003e@\u003cnone\u003e"],"sizeBytes":116704924},{"names":["gcr.io/google_containers/kube-controller-manager-arm:v1.4.0"],"sizeBytes":106962874},{"names":["gcr.io/google_containers/kube-scheduler-arm:v1.4.0"],"sizeBytes":63165102},{"names":["quay.io/coreos/flannel-git:v0.6.1-28-g5dde68d-arm"],"sizeBytes":59864639},{"names":["gcr.io/google_containers/etcd-arm:2.2.5"],"sizeBytes":24905124},{"names":["armel/busybox:latest"],"sizeBytes":3410796},{"names":["gcr.io/google_containers/pause-arm:3.0"],"sizeBytes":506244}]}}]}
I1007 10:32:31.184692       1 round_trippers.go:299] curl -k -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: flanneld/v1.4.0 (linux/arm) kubernetes/$Format" -H "Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkZWZhdWx0LXRva2VuLTM4MmgyIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImRlZmF1bHQiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiIxZmNiYzgwNC04Yzc4LTExZTYtOTBiOS1iODI3ZWI2NDUzNzQiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZS1zeXN0ZW06ZGVmYXVsdCJ9.PpEau6WesQfvLhQ560ioXsiYqmtE-oH9smCbMeX8kO9bi2M_nKlUxSmgbLCk1-jh0FhrpK1n_qDSTqdk1UGmNM8DB1CdJwRH1QaiVmwo9k0yn4tU-I10MB7jQHzLJAlKKCoFR0nwzxPTTdRdcHvW3sz_0h8GhLOF4tdWOv70qO_UfqGjDCQrHR04RScn-pa4R1tUiisnPpFkI3H0l5k2qJHpoWC4xzGA83H-flLzScHpIIdCN0o6armiSNplF17LOPvK6XW5Z6PSaZBGnI5-zb3GJMK7JIJHArE7Bs_EZqCoz-kKryUIpVeL8i84Y0XVcGK3rg-4ogaHPYYx1d70EA" https://100.64.0.1:443/api/v1/watch/nodes?resourceVersion=1873&timeoutSeconds=481
I1007 10:32:31.194832       1 round_trippers.go:318] GET https://100.64.0.1:443/api/v1/watch/nodes?resourceVersion=1873&timeoutSeconds=481 200 OK in 10 milliseconds
I1007 10:32:31.197110       1 round_trippers.go:324] Response Headers:
I1007 10:32:31.197222       1 round_trippers.go:327]     Content-Type: application/json
I1007 10:32:31.197267       1 round_trippers.go:327]     Date: Fri, 07 Oct 2016 10:32:31 GMT
E1007 10:32:32.180297       1 network.go:106] failed to register network: operation not supported
E1007 10:32:33.185074       1 network.go:106] failed to register network: operation not supported

[luxas]

@tomdee @mlbiam Can you verify CONFIG_VXLAN is enabled when running

curl -sSL https://raw.githubusercontent.com/docker/docker/master/contrib/check-config.sh | bash

I think the problem is that vxlan support isn't present
Also, try going down to udp and see if it works

[mlbiam]

I actually did try udp with another error. Will try this afternoon to get
you the error

On Oct 8, 2016 10:51 AM, "Lucas Käldström" [email protected] wrote:

@tomdee https://github.com/tomdee @mlbiam https://github.com/mlbiam
Can you verify CONFIG_VXLAN is enabled when running

curl -sSL https://raw.githubusercontent.com/docker/docker/master/contrib/check-config.sh | bash

I think the problem is that vxlan support isn't present
Also, try going down to udp and see if it works

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/coreos/flannel/issues/512#issuecomment-252428662, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AH3fw10Nn1EJ-uGCQIl5zr9idGdnBlX9ks5qx64EgaJpZM4KDZME
.

[mlbiam]

@luxas the script didn't seem to work:

I changed the backend to udp and now am getting:

root@k8s-pi-master:~# curl -sSL https://raw.githubusercontent.com/docker/docker/master/contrib/check-config.sh | bash
warning: /proc/config.gz does not exist, searching other paths for kernel config ...
error: cannot find kernel config
  try running this script again, specifying the kernel config:
    CONFIG=/path/to/kernel/.config bash or bash /path/to/kernel/.config

root@k8s-pi-master:~# kubectl logs kube-flannel-ds-4f95s kube-flannel  --namespace=kube-system
I1008 19:38:47.932625       1 round_trippers.go:299] curl -k -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: flanneld/v1.4.0 (linux/arm) kubernetes/$Format" -H "Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkZWZhdWx0LXRva2VuLWdyOHUwIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImRlZmF1bHQiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJjMjZjZTE0Ni04YzkyLTExZTYtYmIxZC1iODI3ZWI2NDUzNzQiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZS1zeXN0ZW06ZGVmYXVsdCJ9.ZYNQMgq7rR0uQUf2Mo_pEkWAlbN0rF047-y_S_dXfdZlV76yfDRfd3xJoygVBb3HMXvtJVcM4P12h9G5uJ6oIWnJPQ_rd6SwUJtznkqwlPSyjKuZMPhe7COow3i0Kh7T1WQd_MffUm6MuxGMrp9SnnvQmt14n3C8uB3tq3yqQoRdoMjWzRe33iJKYhBXsiczeG14b82-LMn-qyMRuMVYFZ8aG3J-wGL70VBTbJWEZCbrQtC03aUzxJF2WEcpej4P9ck1oYtM9Ed3uxNI2FFlpa0vSfjIcV6hxdlZrcVRqMfae4_W2uPU737NUyPqEpcSmU8oj9Mj9X27EtvnuqaDBQ" https://100.64.0.1:443/api/v1/namespaces/kube-system/pods/kube-flannel-ds-4f95s
I1008 19:38:48.350511       1 round_trippers.go:318] GET https://100.64.0.1:443/api/v1/namespaces/kube-system/pods/kube-flannel-ds-4f95s 200 OK in 416 milliseconds
I1008 19:38:48.350779       1 round_trippers.go:324] Response Headers:
I1008 19:38:48.350820       1 round_trippers.go:327]     Content-Type: application/json
I1008 19:38:48.351051       1 round_trippers.go:327]     Date: Sat, 08 Oct 2016 19:38:48 GMT
I1008 19:38:48.353068       1 request.go:908] Response Body: {"kind":"Pod","apiVersion":"v1","metadata":{"name":"kube-flannel-ds-4f95s","generateName":"kube-flannel-ds-","namespace":"kube-system","selfLink":"/api/v1/namespaces/kube-system/pods/kube-flannel-ds-4f95s","uid":"d37e6451-8d8e-11e6-bb1d-b827eb645374","resourceVersion":"168311","creationTimestamp":"2016-10-08T19:38:45Z","labels":{"app":"flannel","tier":"node"},"annotations":{"kubernetes.io/created-by":"{\"kind\":\"SerializedReference\",\"apiVersion\":\"v1\",\"reference\":{\"kind\":\"DaemonSet\",\"namespace\":\"kube-system\",\"name\":\"kube-flannel-ds\",\"uid\":\"d3731096-8d8e-11e6-bb1d-b827eb645374\",\"apiVersion\":\"extensions\",\"resourceVersion\":\"168296\"}}\n"}},"spec":{"volumes":[{"name":"run","hostPath":{"path":"/run"}},{"name":"cni","hostPath":{"path":"/etc/cni/net.d"}},{"name":"flannel-cfg","configMap":{"name":"kube-flannel-cfg","defaultMode":420}},{"name":"default-token-gr8u0","secret":{"secretName":"default-token-gr8u0","defaultMode":420}}],"containers":[{"name":"kube-flannel","image":"quay.io/coreos/flannel-git:v0.6.1-28-g5dde68d-arm","command":["/opt/bin/flanneld","--ip-masq","--kube-subnet-mgr","--v=11"],"env":[{"name":"POD_NAME","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"metadata.name"}}},{"name":"POD_NAMESPACE","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"metadata.namespace"}}}],"resources":{},"volumeMounts":[{"name":"run","mountPath":"/run"},{"name":"flannel-cfg","mountPath":"/etc/kube-flannel/"},{"name":"default-token-gr8u0","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","imagePullPolicy":"IfNotPresent","securityContext":{"privileged":true}},{"name":"install-cni","image":"quay.io/coreos/flannel-git:v0.6.1-28-g5dde68d-arm","command":["/bin/sh","-c","set -e -x; cp -f /etc/kube-flannel/cni-conf.json /etc/cni/net.d/10-flannel.conf; while true; do sleep 3600; done"],"resources":{},"volumeMounts":[{"name":"cni","mountPath":"/etc/cni/net.d"},{"name":"flannel-cfg","mountPath":"/etc/kube-flannel/"},{"name":"default-token-gr8u0","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","nodeSelector":{"beta.kubernetes.io/arch":"arm"},"serviceAccountName":"default","serviceAccount":"default","nodeName":"k8s-pi-master","hostNetwork":true,"securityContext":{}},"status":{"phase":"Running","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2016-10-08T19:38:45Z"},{"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":"2016-10-08T19:38:47Z"}],"hostIP":"192.168.3.50","podIP":"192.168.3.50","startTime":"2016-10-08T19:38:45Z","containerStatuses":[{"name":"install-cni","state":{"running":{"startedAt":"2016-10-08T19:38:47Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"quay.io/coreos/flannel-git:v0.6.1-28-g5dde68d-arm","imageID":"docker://sha256:fc79374db247e8c3fbc9b70dd9eade992ac1794edf5f021db5251180fe6e7451","containerID":"docker://342e37a90ed1d12ccd50a5561f936c076a524d92794a92126882304165ea0e6d"},{"name":"kube-flannel","state":{"running":{"startedAt":"2016-10-08T19:38:46Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"quay.io/coreos/flannel-git:v0.6.1-28-g5dde68d-arm","imageID":"docker://sha256:fc79374db247e8c3fbc9b70dd9eade992ac1794edf5f021db5251180fe6e7451","containerID":"docker://7731e94c3eb63d76fb63b8468474217eaf9d9f16356d4f01ff829eefc8100f1e"}]}}
I1008 19:38:48.390595       1 main.go:132] Installing signal handlers
I1008 19:38:48.390882       1 kube.go:233] starting kube subnet manager
I1008 19:38:48.391704       1 manager.go:133] Determining IP address of default interface
I1008 19:38:48.399537       1 manager.go:163] Using 192.168.3.50 as external interface
I1008 19:38:48.399633       1 manager.go:164] Using 192.168.3.50 as external endpoint
E1008 19:38:48.400105       1 network.go:106] failed to register network: failed to acquire lease: node "k8s-pi-master" not found
I1008 19:38:48.392403       1 reflector.go:211] Starting reflector *api.Node (5m0s) from github.com/coreos/flannel/subnet/kube/kube.go:234
I1008 19:38:48.400626       1 reflector.go:249] Listing and watching *api.Node from github.com/coreos/flannel/subnet/kube/kube.go:234
I1008 19:38:48.413962       1 round_trippers.go:299] curl -k -v -XGET  -H "User-Agent: flanneld/v1.4.0 (linux/arm) kubernetes/$Format" -H "Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkZWZhdWx0LXRva2VuLWdyOHUwIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImRlZmF1bHQiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJjMjZjZTE0Ni04YzkyLTExZTYtYmIxZC1iODI3ZWI2NDUzNzQiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZS1zeXN0ZW06ZGVmYXVsdCJ9.ZYNQMgq7rR0uQUf2Mo_pEkWAlbN0rF047-y_S_dXfdZlV76yfDRfd3xJoygVBb3HMXvtJVcM4P12h9G5uJ6oIWnJPQ_rd6SwUJtznkqwlPSyjKuZMPhe7COow3i0Kh7T1WQd_MffUm6MuxGMrp9SnnvQmt14n3C8uB3tq3yqQoRdoMjWzRe33iJKYhBXsiczeG14b82-LMn-qyMRuMVYFZ8aG3J-wGL70VBTbJWEZCbrQtC03aUzxJF2WEcpej4P9ck1oYtM9Ed3uxNI2FFlpa0vSfjIcV6hxdlZrcVRqMfae4_W2uPU737NUyPqEpcSmU8oj9Mj9X27EtvnuqaDBQ" -H "Accept: application/json, */*" https://100.64.0.1:443/api/v1/nodes?resourceVersion=0
I1008 19:38:48.423491       1 round_trippers.go:318] GET https://100.64.0.1:443/api/v1/nodes?resourceVersion=0 200 OK in 9 milliseconds
I1008 19:38:48.423609       1 round_trippers.go:324] Response Headers:
I1008 19:38:48.423643       1 round_trippers.go:327]     Content-Type: application/json
I1008 19:38:48.423677       1 round_trippers.go:327]     Date: Sat, 08 Oct 2016 19:38:48 GMT
I1008 19:38:48.424067       1 request.go:908] Response Body: {"kind":"NodeList","apiVersion":"v1","metadata":{"selfLink":"/api/v1/nodes","resourceVersion":"168302"},"items":[{"metadata":{"name":"k8s-pi-master","selfLink":"/api/v1/nodes/k8s-pi-master","uid":"c1ea32bf-8c92-11e6-bb1d-b827eb645374","resourceVersion":"168302","creationTimestamp":"2016-10-07T13:34:22Z","labels":{"beta.kubernetes.io/arch":"arm","beta.kubernetes.io/os":"linux","kubeadm.alpha.kubernetes.io/role":"master","kubernetes.io/hostname":"k8s-pi-master"},"annotations":{"scheduler.alpha.kubernetes.io/taints":"[{\"key\":\"dedicated\",\"value\":\"master\",\"effect\":\"NoSchedule\"}]","volumes.kubernetes.io/controller-managed-attach-detach":"true"}},"spec":{"externalID":"k8s-pi-master"},"status":{"capacity":{"alpha.kubernetes.io/nvidia-gpu":"0","cpu":"4","memory":"948012Ki","pods":"110"},"allocatable":{"alpha.kubernetes.io/nvidia-gpu":"0","cpu":"4","memory":"948012Ki","pods":"110"},"conditions":[{"type":"OutOfDisk","status":"False","lastHeartbeatTime":"2016-10-08T19:38:46Z","lastTransitionTime":"2016-10-07T13:34:22Z","reason":"KubeletHasSufficientDisk","message":"kubelet has sufficient disk space available"},{"type":"MemoryPressure","status":"False","lastHeartbeatTime":"2016-10-08T19:38:46Z","lastTransitionTime":"2016-10-07T13:34:22Z","reason":"KubeletHasSufficientMemory","message":"kubelet has sufficient memory available"},{"type":"DiskPressure","status":"False","lastHeartbeatTime":"2016-10-08T19:38:46Z","lastTransitionTime":"2016-10-07T13:34:22Z","reason":"KubeletHasNoDiskPressure","message":"kubelet has no disk pressure"},{"type":"Ready","status":"True","lastHeartbeatTime":"2016-10-08T19:38:46Z","lastTransitionTime":"2016-10-07T13:34:22Z","reason":"KubeletReady","message":"kubelet is posting ready status. WARNING: CPU hardcapping unsupported"}],"addresses":[{"type":"LegacyHostIP","address":"192.168.3.50"},{"type":"InternalIP","address":"192.168.3.50"}],"daemonEndpoints":{"kubeletEndpoint":{"Port":10250}},"nodeInfo":{"machineID":"f996bd096b8f418da602c292906fa243","systemUUID":"f996bd096b8f418da602c292906fa243","bootID":"c9375c4c-2480-41ba-ab38-b47e53ce6d1b","kernelVersion":"4.1.19-v7+","osImage":"Ubuntu 16.04 LTS","containerRuntimeVersion":"docker://1.11.2","kubeletVersion":"v1.4.0","kubeProxyVersion":"v1.4.0","operatingSystem":"linux","architecture":"arm"},"images":[{"names":["gcr.io/google_containers/kube-proxy-arm:v1.4.0"],"sizeBytes":177088263},{"names":["gcr.io/google_containers/kube-discovery-arm:1.0"],"sizeBytes":122116943},{"names":["gcr.io/google_containers/kube-apiserver-arm:v1.4.0"],"sizeBytes":116704924},{"names":["gcr.io/google_containers/kube-controller-manager-arm:v1.4.0"],"sizeBytes":106962874},{"names":["gcr.io/google_containers/kube-scheduler-arm:v1.4.0"],"sizeBytes":63165102},{"names":["quay.io/coreos/flannel-git:v0.6.1-28-g5dde68d-arm"],"sizeBytes":59864639},{"names":["gcr.io/google_containers/etcd-arm:2.2.5"],"sizeBytes":24905124},{"names":["armel/busybox:latest"],"sizeBytes":3410796},{"names":["gcr.io/google_containers/pause-arm:3.0"],"sizeBytes":506244}]}}]}
I1008 19:38:48.432691       1 round_trippers.go:299] curl -k -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: flanneld/v1.4.0 (linux/arm) kubernetes/$Format" -H "Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkZWZhdWx0LXRva2VuLWdyOHUwIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImRlZmF1bHQiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJjMjZjZTE0Ni04YzkyLTExZTYtYmIxZC1iODI3ZWI2NDUzNzQiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZS1zeXN0ZW06ZGVmYXVsdCJ9.ZYNQMgq7rR0uQUf2Mo_pEkWAlbN0rF047-y_S_dXfdZlV76yfDRfd3xJoygVBb3HMXvtJVcM4P12h9G5uJ6oIWnJPQ_rd6SwUJtznkqwlPSyjKuZMPhe7COow3i0Kh7T1WQd_MffUm6MuxGMrp9SnnvQmt14n3C8uB3tq3yqQoRdoMjWzRe33iJKYhBXsiczeG14b82-LMn-qyMRuMVYFZ8aG3J-wGL70VBTbJWEZCbrQtC03aUzxJF2WEcpej4P9ck1oYtM9Ed3uxNI2FFlpa0vSfjIcV6hxdlZrcVRqMfae4_W2uPU737NUyPqEpcSmU8oj9Mj9X27EtvnuqaDBQ" https://100.64.0.1:443/api/v1/watch/nodes?resourceVersion=168302&timeoutSeconds=481
I1008 19:38:48.439050       1 round_trippers.go:318] GET https://100.64.0.1:443/api/v1/watch/nodes?resourceVersion=168302&timeoutSeconds=481 200 OK in 6 milliseconds
I1008 19:38:48.439245       1 round_trippers.go:324] Response Headers:
I1008 19:38:48.439282       1 round_trippers.go:327]     Content-Type: application/json
I1008 19:38:48.439337       1 round_trippers.go:327]     Date: Sat, 08 Oct 2016 19:38:48 GMT
E1008 19:38:49.400929       1 network.go:106] failed to register network: failed to acquire lease: node "k8s-pi-master" pod cidr not assigned

[mlbiam]

@luxas @tomdee

I think I got this working! I added "--pod-network-cidr" to kubeadm:

$ kubeadm init --pod-network-cidr 10.244.0.0/16

I also replaced vxlan with udp. Now all my pods are working (even past a reboot). on to getting a minion running!

[luxas]

Oops, I did that all the time and it worked for me, sorry for not informing good enough that you should have done that.

Well, seems like you did figure it out :)

[luxas]

Anyway, for others encountering this problem:

The --allocate-node-cidrs=true and --cluster-cidr=10.244.0.0/16 flags have to be set on controller-manager for it to function

[mlbiam]

I certainly learned more about how Kubernetes networking works!

[autostatic]

Exact same issue here. Unfortunately I still get the message below on my nodes (minions) despite having changed to UDP and setting --pod-network-cidr with kubeadm init:
failed to register network: failed to acquire lease: node "some.k8s.node" pod cidr not assigned

[cizixs]

Found the documentation for this:

The flannel kube subnet manager relies on the fact that each node already has a podCIDR defined.
If your nodes do not have a podCIDR, then either use the --pod-cidr kubelet command-line option or the --allocate-node-cidrs=true --cluster-cidr=<cidr> controller-manager command-line options.

If kubeadm is being used then pass --pod-network-cidr=10.244.0.0/16 to kubeadm init which will ensure that all nodes are automatically assigned a podCIDR.

source: https://coreos.com/flannel/docs/latest/troubleshooting.html