Attempting to run the Origin docker container using the command listed in the docs fails with the error in the title.
N/A, haven't even gotten far enough for it to work
sudo docker run -d --name "origin" \
--privileged --pid=host --net=host \
-v /:/rootfs:ro -v /var/run:/var/run:rw -v /sys:/sys -v /sys/fs/cgroup:/sys/fs/cgroup:rw \
-v /var/lib/docker:/var/lib/docker:rw \
openshift/origin:latest start
docker logs -f origin and wait for output to finishF0620 08:05:16.627956 6766 node.go:297] failed to run Kubelet: failed to create kubelet: misconfiguration: kubelet cgroup driver: "systemd" is different from docker cgroup driver: "cgroupfs"
Container should start up without issues
lsb_release -a:
user@host ~> lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux 8.8 (jessie)
Release: 8.8
Codename: jessie
docker -v:
user@host ~> docker -v
Docker version 17.03.1-ce, build c6d412e
Full container logs:
user@host ~> docker logs -f origin
W0620 08:04:59.197432 6766 start_master.go:291] Warning: assetConfig.loggingPublicURL: Invalid value: "": required to view aggregated container logs in the console, master start will continue.
W0620 08:04:59.197807 6766 start_master.go:291] Warning: assetConfig.metricsPublicURL: Invalid value: "": required to view cluster metrics in the console, master start will continue.
W0620 08:04:59.197840 6766 start_master.go:291] Warning: auditConfig.auditFilePath: Required value: audit can not be logged to a separate file, master start will continue.
I0620 08:04:59.213516 6766 plugins.go:101] No cloud provider specified.
2017-06-20 08:04:59.224894 I | etcdserver/api/v3rpc: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp AAA.BBB.CCC.DDD:4001: getsockopt: connection refused"; Reconnecting to {AAA.BBB.CCC.DDD:4001 <nil>}
2017-06-20 08:04:59.225081 I | etcdserver/api/v3rpc: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp AAA.BBB.CCC.DDD:4001: getsockopt: connection refused"; Reconnecting to {AAA.BBB.CCC.DDD:4001 <nil>}
2017-06-20 08:04:59.225164 I | etcdserver/api/v3rpc: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp AAA.BBB.CCC.DDD:4001: getsockopt: connection refused"; Reconnecting to {AAA.BBB.CCC.DDD:4001 <nil>}
2017-06-20 08:04:59.225234 I | etcdserver/api/v3rpc: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp AAA.BBB.CCC.DDD:4001: getsockopt: connection refused"; Reconnecting to {AAA.BBB.CCC.DDD:4001 <nil>}
2017-06-20 08:04:59.244676 I | etcdserver/api/v3rpc: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp AAA.BBB.CCC.DDD:4001: getsockopt: connection refused"; Reconnecting to {AAA.BBB.CCC.DDD:4001 <nil>}
I0620 08:04:59.578567 6766 start_master.go:430] Starting master on 0.0.0.0:8443 (v3.6.0-alpha.2+2d90d52-359)
I0620 08:04:59.578670 6766 start_master.go:431] Public master address is https://AAA.BBB.CCC.DDD:8443
I0620 08:04:59.578728 6766 start_master.go:435] Using images from "openshift/origin-<component>:v3.6.0-alpha.2"
2017-06-20 08:04:59.578989 I | embed: peerTLS: cert = openshift.local.config/master/etcd.server.crt, key = openshift.local.config/master/etcd.server.key, ca = openshift.local.config/master/ca.crt, trusted-ca = , client-cert-auth = true
2017-06-20 08:04:59.579788 I | embed: listening for peers on https://0.0.0.0:7001
2017-06-20 08:04:59.579874 I | embed: listening for client requests on 0.0.0.0:4001
2017-06-20 08:04:59.593344 I | etcdserver/api/v3rpc: grpc: addrConn.resetTransport failed to create client transport: connection error: desc = "transport: dial tcp AAA.BBB.CCC.DDD:4001: getsockopt: connection refused"; Reconnecting to {AAA.BBB.CCC.DDD:4001 <nil>}
2017-06-20 08:04:59.599489 I | etcdserver: name = openshift.local
2017-06-20 08:04:59.599573 I | etcdserver: data dir = openshift.local.etcd
2017-06-20 08:04:59.599595 I | etcdserver: member dir = openshift.local.etcd/member
2017-06-20 08:04:59.599612 I | etcdserver: heartbeat = 100ms
2017-06-20 08:04:59.599642 I | etcdserver: election = 1000ms
2017-06-20 08:04:59.599657 I | etcdserver: snapshot count = 100000
2017-06-20 08:04:59.599681 I | etcdserver: advertise client URLs = https://AAA.BBB.CCC.DDD:4001
2017-06-20 08:04:59.599714 I | etcdserver: initial advertise peer URLs = https://AAA.BBB.CCC.DDD:7001
2017-06-20 08:04:59.599739 I | etcdserver: initial cluster = openshift.local=https://AAA.BBB.CCC.DDD:7001
2017-06-20 08:04:59.604307 I | etcdserver: starting member d2ca0e9a4ad0cca7 in cluster dda1658ef20fdb45
2017-06-20 08:04:59.604538 I | raft: d2ca0e9a4ad0cca7 became follower at term 0
2017-06-20 08:04:59.604602 I | raft: newRaft d2ca0e9a4ad0cca7 [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]
2017-06-20 08:04:59.604678 I | raft: d2ca0e9a4ad0cca7 became follower at term 1
2017-06-20 08:04:59.610276 W | auth: simple token is not cryptographically signed
2017-06-20 08:04:59.622067 I | etcdserver: starting server... [version: 3.2.0, cluster version: to_be_decided]
2017-06-20 08:04:59.622182 I | embed: ClientTLS: cert = openshift.local.config/master/etcd.server.crt, key = openshift.local.config/master/etcd.server.key, ca = openshift.local.config/master/ca.crt, trusted-ca = , client-cert-auth = true
2017-06-20 08:04:59.643352 I | etcdserver/membership: added member d2ca0e9a4ad0cca7 [https://AAA.BBB.CCC.DDD:7001] to cluster dda1658ef20fdb45
2017-06-20 08:05:00.605265 I | raft: d2ca0e9a4ad0cca7 is starting a new election at term 1
2017-06-20 08:05:00.605394 I | raft: d2ca0e9a4ad0cca7 became candidate at term 2
2017-06-20 08:05:00.605466 I | raft: d2ca0e9a4ad0cca7 received MsgVoteResp from d2ca0e9a4ad0cca7 at term 2
2017-06-20 08:05:00.605492 I | raft: d2ca0e9a4ad0cca7 became leader at term 2
2017-06-20 08:05:00.605500 I | raft: raft.node: d2ca0e9a4ad0cca7 elected leader d2ca0e9a4ad0cca7 at term 2
2017-06-20 08:05:00.606065 I | etcdserver: setting up the initial cluster version to 3.2
2017-06-20 08:05:00.606148 I | etcdserver: published {Name:openshift.local ClientURLs:[https://AAA.BBB.CCC.DDD:4001]} to cluster dda1658ef20fdb45
2017-06-20 08:05:00.606161 I | embed: ready to serve client requests
2017-06-20 08:05:00.606843 I | embed: serving client requests on [::]:4001
2017-06-20 08:05:00.662051 N | etcdserver/membership: set the initial cluster version to 3.2
2017-06-20 08:05:00.662431 I | etcdserver/api: enabled capabilities for version 3.2
I0620 08:05:00.699501 6766 run.go:85] Started etcd at AAA.BBB.CCC.DDD:4001
2017-06-20 08:05:00.803943 I | etcdserver/api/v3rpc: Failed to dial [::]:4001: connection error: desc = "transport: remote error: tls: bad certificate"; please retry.
2017-06-20 08:05:00.804268 I | etcdserver/api/v3rpc: Failed to dial [::]:4001: connection error: desc = "transport: remote error: tls: bad certificate"; please retry.
2017-06-20 08:05:00.804356 I | etcdserver/api/v3rpc: Failed to dial [::]:4001: connection error: desc = "transport: remote error: tls: bad certificate"; please retry.
2017-06-20 08:05:00.804462 I | etcdserver/api/v3rpc: Failed to dial [::]:4001: connection error: desc = "transport: remote error: tls: bad certificate"; please retry.
2017-06-20 08:05:00.807063 I | etcdserver/api/v3rpc: Failed to dial [::]:4001: connection error: desc = "transport: remote error: tls: bad certificate"; please retry.
2017-06-20 08:05:00.807279 I | etcdserver/api/v3rpc: Failed to dial [::]:4001: connection error: desc = "transport: remote error: tls: bad certificate"; please retry.
2017-06-20 08:05:00.811270 I | etcdserver/api/v3rpc: Failed to dial [::]:4001: connection error: desc = "transport: remote error: tls: bad certificate"; please retry.
2017-06-20 08:05:00.811361 I | etcdserver/api/v3rpc: Failed to dial [::]:4001: connection error: desc = "transport: remote error: tls: bad certificate"; please retry.
I0620 08:05:00.818752 6766 run_components.go:113] Using default project node label selector:
I0620 08:05:00.822754 6766 master.go:169] Starting OAuth2 API at /oauth
I0620 08:05:00.822815 6766 master.go:177] Starting Web Console /console/
I0620 08:05:00.826591 6766 clusterquotamapping.go:101] Starting ClusterQuotaMappingController controller
E0620 08:05:00.830459 6766 reflector.go:201] github.com/openshift/origin/pkg/authorization/generated/informers/internalversion/factory.go:45: Failed to list *api.PolicyBinding: Get https://AAA.BBB.CCC.DDD:8443/apis/authorization.openshift.io/v1/policybindings?resourceVersion=0: dial tcp AAA.BBB.CCC.DDD:8443: getsockopt: connection refused
E0620 08:05:00.835261 6766 reflector.go:201] github.com/openshift/origin/pkg/authorization/generated/informers/internalversion/factory.go:45: Failed to list *api.Policy: Get https://AAA.BBB.CCC.DDD:8443/apis/authorization.openshift.io/v1/policies?resourceVersion=0: dial tcp AAA.BBB.CCC.DDD:8443: getsockopt: connection refused
E0620 08:05:00.835539 6766 reflector.go:201] github.com/openshift/origin/pkg/authorization/generated/informers/internalversion/factory.go:45: Failed to list *api.ClusterPolicyBinding: Get https://AAA.BBB.CCC.DDD:8443/apis/authorization.openshift.io/v1/clusterpolicybindings?resourceVersion=0: dial tcp AAA.BBB.CCC.DDD:8443: getsockopt: connection refused
E0620 08:05:00.836570 6766 reflector.go:201] github.com/openshift/origin/pkg/authorization/generated/informers/internalversion/factory.go:45: Failed to list *api.ClusterPolicy: Get https://AAA.BBB.CCC.DDD:8443/apis/authorization.openshift.io/v1/clusterpolicies?resourceVersion=0: dial tcp AAA.BBB.CCC.DDD:8443: getsockopt: connection refused
E0620 08:05:00.837038 6766 reflector.go:201] github.com/openshift/origin/pkg/quota/generated/informers/internalversion/factory.go:45: Failed to list *api.ClusterResourceQuota: Get https://AAA.BBB.CCC.DDD:8443/apis/quota.openshift.io/v1/clusterresourcequotas?resourceVersion=0: dial tcp AAA.BBB.CCC.DDD:8443: getsockopt: connection refused
W0620 08:05:01.043705 6766 genericapiserver.go:295] Skipping API rbac.authorization.k8s.io/v1alpha1 because it has no resources.
I0620 08:05:01.314079 6766 openshift-apiserver.go:227] Starting Origin API at /apis/project.openshift.io/v1
I0620 08:05:01.317611 6766 openshift-apiserver.go:227] Starting Origin API at /apis/build.openshift.io/v1
I0620 08:05:01.324881 6766 openshift-apiserver.go:227] Starting Origin API at /apis/authorization.openshift.io/v1
I0620 08:05:01.328073 6766 openshift-apiserver.go:227] Starting Origin API at /apis/oauth.openshift.io/v1
I0620 08:05:01.329829 6766 openshift-apiserver.go:227] Starting Origin API at /apis/security.openshift.io/v1
I0620 08:05:01.558613 6766 openshift-apiserver.go:227] Starting Origin API at /apis/network.openshift.io/v1
I0620 08:05:01.561076 6766 openshift-apiserver.go:227] Starting Origin API at /apis/route.openshift.io/v1
I0620 08:05:01.563838 6766 openshift-apiserver.go:227] Starting Origin API at /apis/user.openshift.io/v1
I0620 08:05:01.567817 6766 openshift-apiserver.go:227] Starting Origin API at /apis/image.openshift.io/v1
I0620 08:05:01.570523 6766 openshift-apiserver.go:227] Starting Origin API at /apis/apps.openshift.io/v1
I0620 08:05:01.572260 6766 openshift-apiserver.go:227] Starting Origin API at /apis/template.openshift.io/v1
I0620 08:05:01.755206 6766 openshift-apiserver.go:227] Starting Origin API at /apis/quota.openshift.io/v1
I0620 08:05:01.928413 6766 openshift-apiserver.go:233] Started Origin API at /oapi/v1
E0620 08:05:01.970260 6766 reflector.go:201] github.com/openshift/origin/pkg/quota/generated/informers/internalversion/factory.go:45: Failed to list *api.ClusterResourceQuota: Get https://AAA.BBB.CCC.DDD:8443/apis/quota.openshift.io/v1/clusterresourcequotas?resourceVersion=0: dial tcp AAA.BBB.CCC.DDD:8443: getsockopt: connection refused
E0620 08:05:01.991911 6766 reflector.go:201] github.com/openshift/origin/pkg/authorization/generated/informers/internalversion/factory.go:45: Failed to list *api.ClusterPolicy: Get https://AAA.BBB.CCC.DDD:8443/apis/authorization.openshift.io/v1/clusterpolicies?resourceVersion=0: dial tcp AAA.BBB.CCC.DDD:8443: getsockopt: connection refused
E0620 08:05:01.992224 6766 reflector.go:201] github.com/openshift/origin/pkg/authorization/generated/informers/internalversion/factory.go:45: Failed to list *api.ClusterPolicyBinding: Get https://AAA.BBB.CCC.DDD:8443/apis/authorization.openshift.io/v1/clusterpolicybindings?resourceVersion=0: dial tcp AAA.BBB.CCC.DDD:8443: getsockopt: connection refused
E0620 08:05:01.992345 6766 reflector.go:201] github.com/openshift/origin/pkg/authorization/generated/informers/internalversion/factory.go:45: Failed to list *api.Policy: Get https://AAA.BBB.CCC.DDD:8443/apis/authorization.openshift.io/v1/policies?resourceVersion=0: dial tcp AAA.BBB.CCC.DDD:8443: getsockopt: connection refused
E0620 08:05:01.993627 6766 reflector.go:201] github.com/openshift/origin/pkg/authorization/generated/informers/internalversion/factory.go:45: Failed to list *api.PolicyBinding: Get https://AAA.BBB.CCC.DDD:8443/apis/authorization.openshift.io/v1/policybindings?resourceVersion=0: dial tcp AAA.BBB.CCC.DDD:8443: getsockopt: connection refused
[restful] 2017/06/20 08:05:02 log.go:30: [restful/swagger] listing is available at https://AAA.BBB.CCC.DDD:8443/swaggerapi
[restful] 2017/06/20 08:05:02 log.go:30: [restful/swagger] https://AAA.BBB.CCC.DDD:8443/swaggerui/ is mapped to folder /swagger-ui/
I0620 08:05:02.483181 6766 serve.go:86] Serving securely on 0.0.0.0:8443
W0620 08:05:02.598837 6766 run_components.go:82] Binding DNS on port 8053 instead of 53, which may not be resolvable from all clients
I0620 08:05:02.600351 6766 logs.go:41] skydns: ready for queries on cluster.local. for tcp4://0.0.0.0:8053 [rcache 0]
I0620 08:05:02.600419 6766 logs.go:41] skydns: ready for queries on cluster.local. for udp4://0.0.0.0:8053 [rcache 0]
I0620 08:05:02.701612 6766 run_components.go:108] DNS listening at 0.0.0.0:8053
I0620 08:05:02.737281 6766 ensure.go:223] No cluster policy found. Creating bootstrap policy based on: openshift.local.config/master/policy.json
I0620 08:05:03.356618 6766 trace.go:61] Trace "Create /api/v1/namespaces/default/services" (started 2017-06-20 08:05:02.529547434 +0000 UTC):
[18.731碌s] [18.731碌s] About to convert to expected version
[74.247碌s] [55.516碌s] Conversion done
[668.475248ms] [668.401001ms] About to store object in database
[826.958626ms] [158.483378ms] Object stored in database
[826.960666ms] [2.04碌s] Self-link added
"Create /api/v1/namespaces/default/services" [826.994112ms] [33.446碌s] END
W0620 08:05:03.377338 6766 lease_endpoint_reconciler.go:176] Resetting endpoints for master service "kubernetes" to [AAA.BBB.CCC.DDD]
I0620 08:05:13.208523 6766 ensure.go:208] Created default security context constraint privileged
I0620 08:05:13.213550 6766 ensure.go:208] Created default security context constraint nonroot
I0620 08:05:13.218318 6766 ensure.go:208] Created default security context constraint hostmount-anyuid
I0620 08:05:13.221744 6766 ensure.go:208] Created default security context constraint hostaccess
I0620 08:05:13.226588 6766 ensure.go:208] Created default security context constraint restricted
I0620 08:05:13.230435 6766 ensure.go:208] Created default security context constraint anyuid
I0620 08:05:13.235607 6766 ensure.go:208] Created default security context constraint hostnetwork
I0620 08:05:14.599503 6766 start_master.go:643] Controllers starting (*)
I0620 08:05:15.432208 6766 docker.go:364] Connecting to docker on unix:///var/run/docker.sock
I0620 08:05:15.433072 6766 docker.go:384] Start docker client with request timeout=2m0s
W0620 08:05:15.436757 6766 cni.go:157] Unable to update cni config: No networks found in /etc/cni/net.d
I0620 08:05:15.525973 6766 start_master.go:728] Started serviceaccount-token controller
I0620 08:05:15.586129 6766 nodecontroller.go:219] Sending events to api server.
I0620 08:05:15.587324 6766 taint_controller.go:157] Sending events to api server.
I0620 08:05:15.589358 6766 node_config.go:370] DNS Bind to AAA.BBB.CCC.DDD:53
I0620 08:05:15.589428 6766 start_node.go:345] Starting node subdomain.example.com (v3.6.0-alpha.2+2d90d52-359)
I0620 08:05:15.592536 6766 start_node.go:354] Connecting to API server https://AAA.BBB.CCC.DDD:8443
W0620 08:05:15.602947 6766 node.go:207] Error running 'chcon' to set the kubelet volume root directory SELinux context: exit status 1
I0620 08:05:15.603882 6766 docker.go:364] Connecting to docker on unix:///var/run/docker.sock
I0620 08:05:15.603959 6766 docker.go:384] Start docker client with request timeout=2m0s
I0620 08:05:15.608202 6766 node.go:143] Connecting to Docker at unix:///var/run/docker.sock
I0620 08:05:15.655371 6766 feature_gate.go:144] feature gates: map[]
I0620 08:05:15.656607 6766 manager.go:143] cAdvisor running in container: "/docker/75fec75eb7a94c61a55caef95332c5609d2cbfa4e427c46fc2a4688a0851cd4b"
I0620 08:05:15.681562 6766 node.go:364] Using iptables Proxier.
W0620 08:05:15.719020 6766 manager.go:151] unable to connect to Rkt api service: rkt: cannot tcp Dial rkt api service: dial tcp 127.0.0.1:15441: getsockopt: connection refused
W0620 08:05:15.724558 6766 node.go:501] Failed to retrieve node info: nodes "subdomain.example.com" not found
W0620 08:05:15.724655 6766 proxier.go:309] invalid nodeIP, initializing kube-proxy with 127.0.0.1 as nodeIP
W0620 08:05:15.724663 6766 proxier.go:314] clusterCIDR not specified, unable to distinguish between internal and external traffic
I0620 08:05:15.724688 6766 node.go:393] Tearing down userspace rules.
I0620 08:05:16.018624 6766 fs.go:117] Filesystem partitions: map[/dev/vda1:{mountpoint:/var/lib/docker/aufs major:254 minor:1 fsType:ext3 blockSize:0}]
I0620 08:05:16.021960 6766 manager.go:198] Machine: {NumCores:1 CpuFrequency:2394452 MemoryCapacity:2057629696 MachineID:cdb7f92199c748a7a20d33ab52974cbb SystemUUID:EBF7B3D8-DAB9-47C4-BA0F-284565884230 BootID:447508a8-63c1-42bb-bd41-939ea709f1d3 Filesystems:[{Device:/dev/vda1 DeviceMajor:254 DeviceMinor:1 Capacity:10534313984 Type:vfs Inodes:655360 HasInodes:true} {Device:none DeviceMajor:0 DeviceMinor:43 Capacity:10534313984 Type:vfs Inodes:655360 HasInodes:true}] DiskMap:map[254:0:{Name:vda Major:254 Minor:0 Size:10737418240 Scheduler:none}] NetworkDevices:[{Name:eth0 MacAddress:fa:16:3e:f0:e0:fe Speed:0 Mtu:1500} {Name:tun0 MacAddress: Speed:10 Mtu:1500}] Topology:[{Id:0 Memory:2057629696 Cores:[{Id:0 Threads:[0] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
I0620 08:05:16.152503 6766 manager.go:204] Version: {KernelVersion:3.16.0-4-amd64 ContainerOsVersion:CentOS Linux 7 (Core) DockerVersion:17.03.1-ce DockerAPIVersion:1.27 CadvisorVersion: CadvisorRevision:}
I0620 08:05:16.154539 6766 server.go:509] --cgroups-per-qos enabled, but --cgroup-root was not specified. defaulting to /
I0620 08:05:16.165932 6766 container_manager_linux.go:244] container manager verified user specified cgroup-root exists: /
I0620 08:05:16.166037 6766 container_manager_linux.go:249] Creating Container Manager object based on Node Config: {RuntimeCgroupsName: SystemCgroupsName: KubeletCgroupsName: ContainerRuntime:docker CgroupsPerQOS:true CgroupRoot:/ CgroupDriver:systemd ProtectKernelDefaults:false EnableCRI:true NodeAllocatableConfig:{KubeReservedCgroupName: SystemReservedCgroupName: EnforceNodeAllocatable:map[pods:{}] KubeReserved:map[] SystemReserved:map[] HardEvictionThresholds:[{Signal:memory.available Operator:LessThan Value:{Quantity:100Mi Percentage:0} GracePeriod:0s MinReclaim:<nil>}]} ExperimentalQOSReserved:map[]}
I0620 08:05:16.167827 6766 kubelet.go:265] Watching apiserver
W0620 08:05:16.281243 6766 kubelet_network.go:70] Hairpin mode set to "promiscuous-bridge" but kubenet is not enabled, falling back to "hairpin-veth"
I0620 08:05:16.281363 6766 kubelet.go:494] Hairpin mode set to "hairpin-veth"
W0620 08:05:16.292849 6766 cni.go:157] Unable to update cni config: No networks found in /etc/cni/net.d
I0620 08:05:16.343728 6766 node.go:493] Started Kubernetes Proxy on 0.0.0.0
I0620 08:05:16.346155 6766 node.go:332] Starting DNS on AAA.BBB.CCC.DDD:53
I0620 08:05:16.346912 6766 logs.go:41] skydns: ready for queries on cluster.local. for tcp://AAA.BBB.CCC.DDD:53 [rcache 0]
I0620 08:05:16.346975 6766 logs.go:41] skydns: ready for queries on cluster.local. for udp://AAA.BBB.CCC.DDD:53 [rcache 0]
I0620 08:05:16.509299 6766 docker_service.go:184] Docker cri networking managed by kubernetes.io/no-op
F0620 08:05:16.627956 6766 node.go:297] failed to run Kubelet: failed to create kubelet: misconfiguration: kubelet cgroup driver: "systemd" is different from docker cgroup driver: "cgroupfs"
Kubernetes overrides of the default cfgroups and set to systemd instead. On systems where systemd is not used, this is unfortunate.
I compiled openshift origin again after commenting out line 79 in pkg/cmd/server/kubernetes/node/options/options.go
// server.CgroupDriver = "systemd"
... and now for me docker and kubernetes are using cgroupfs again for me.
Kubernetes does have a method to figure the docker cgroup driver in local-up-cluster.sh. But I don't know how everything is supposed to work together.
CGROUP_DRIVER=$(docker info | grep "Cgroup Driver:" | cut -f3- -d' ')
I get the same error even on Fedora
Fedora release 24 (Twenty Four)
4.13.0-0.rc2.git0.1.fc27.x86_64
docker version
Client:
Version: 17.05.0-ce
API version: 1.29
Go version: go1.7.5
Git commit: 89658be
Built: Thu May 4 22:08:20 2017
OS/Arch: linux/amd64
Server:
Version: 17.05.0-ce
API version: 1.29 (minimum version 1.12)
Go version: go1.7.5
Git commit: 89658be
Built: Thu May 4 22:08:20 2017
OS/Arch: linux/amd64
Experimental: false
node.go:282] failed to run Kubelet: failed to create kubelet: misconfiguration: kubelet cgroup driver: "systemd" is different from docker cgroup driver: "cgroupfs"
Fedora is a frontrunner for SystemD, so if you have installed Docker using any of these methods, you should have a docker with a systemd cgroup driver. run "docker info" and your docker will show cgroupfs as cgroup driver. Probably you are better off reinstalling your docker using these instructions.
https://docs.docker.com/engine/installation/linux/docker-ce/fedora/
Debian is more recently moving towards systemd, but it's optional.
Alpine which I am running my openshift on, doesn't come with systemd. I have to patch the node_config.
@janmg you are right. I had overridden the default system unit file to add a second runtime. And that causes dockerd to switch back to cgroup. Adding systemd explicitly fixes the issue.
The unit file in my case, that works
[Service]
ExecStart=
ExecStart=/usr/bin/dockerd -D --add-runtime cor=/bin/cc-oci-runtime --default-runtime=runc --exec-opt native.cgroupdriver=systemd
[Service]
# Allow maximum number of containers to run.
TasksMax=infinity
openshift version
openshift v3.7.0+7ed6862
kubernetes v1.7.6+a08f5eeb62
etcd 3.2.8
F1213 10:15:19.522499 21099 node.go:264] failed to run Kubelet: failed to create kubelet: misconfiguration: kubelet cgroup driver: "systemd" is different from docker cgroup driver: "cgroupfs"
md5-cc29fce0b935a351386866071ce1bd5c
the default is not to use systemd for cgroups because the delegate issues still
exists and systemd currently does not support the cgroup feature set required
for containers run by docker
F0109 06:13:41.401134 11461 node.go:264] failed to run Kubelet: failed to create kubelet: misconfiguration: kubelet cgroup driver: "systemd" is different from docker cgroup driver: "cgroupfs"
./openshift version
openshift v3.7.0+7ed6862
kubernetes v1.7.6+a08f5eeb62
etcd 3.2.8
md5-cc29fce0b935a351386866071ce1bd5c
docker version
Client:
Version: 17.12.0-ce
API version: 1.35
Go version: go1.9.2
Git commit: c97c6d6
Built: Wed Dec 27 20:10:14 2017
OS/Arch: linux/amd64
Server:
Engine:
Version: 17.12.0-ce
API version: 1.35 (minimum version 1.12)
Go version: go1.9.2
Git commit: c97c6d6
Built: Wed Dec 27 20:12:46 2017
OS/Arch: linux/amd64
Experimental: false
md5-cc29fce0b935a351386866071ce1bd5c
lsb_release -a
LSB Version: :core-4.1-amd64:core-4.1-noarch
Distributor ID: CentOS
Description: CentOS Linux release 7.4.1708 (Core)
Release: 7.4.1708
Codename: Core
can some one give me some advice to fix F0109 06:13:41.401134 11461 node.go:264] failed to run Kubelet: failed to create kubelet: misconfiguration: kubelet cgroup driver: "systemd" is different from docker cgroup driver: "cgroupfs" error?
Same here.
openshift v3.7.1+ab0f056
kubernetes v1.7.6+a08f5eeb62
etcd 3.2.8
Tried on following systems (uname -a):
Linux krna 4.14.13-1-ARCH #1 SMP PREEMPT Wed Jan 10 11:14:50 UTC 2018 x86_64 GNU/LinuxLinux localhost 4.14.11-coreos #1 SMP Fri Jan 5 11:00:14 UTC 2018 x86_64 Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz GenuineIntel GNU/Linux@nazarewk @chenray844 follow @mcastelino advice --exec-opt native.cgroupdriver=systemd
Ran into this exact issue tonight --exec-opt native.cgroupdriver=systemd in /usr/lib/systemd/system/docker.service with: ExecStart=/usr/bin/dockerd --exec-opt native.cgroupdriver=systemd+ docker and kubelet restarts, got this working for me.
Running:
[hostname]# uname -a
Linux hostname.net 3.10.0-327.el7.x86_64 #1 SMP Thu Nov 19 22:10:57 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
[hostname]# cat /etc/redhat-release
CentOS Linux release 7.2.1511 (Core)
Issues go stale after 90d of inactivity.
Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.
If this issue is safe to close now please do so with /close.
/lifecycle stale
Stale issues rot after 30d of inactivity.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.
If this issue is safe to close now please do so with /close.
/lifecycle rotten
/remove-lifecycle stale
Rotten issues close after 30d of inactivity.
Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.
/close
Most helpful comment
@janmg you are right. I had overridden the default system unit file to add a second runtime. And that causes dockerd to switch back to cgroup. Adding systemd explicitly fixes the issue.
The unit file in my case, that works