Cadvisor: cAdvisor:latest fails to start on amazon linux
>docker logs ecs-Monitoring-7-cadvisor-acbff3bae6b0a5803000
I0729 17:24:27.601106 1 storagedriver.go:50] Caching stats in memory for 2m0s
I0729 17:24:27.601446 1 manager.go:135] cAdvisor running in container: "/docker/411895cca1a8a50a51436bd7d760cd4ba890d3a209687341937f8051ea178038"
W0729 17:24:27.620289 1 manager.go:143] unable to connect to Rkt api service: rkt: cannot tcp Dial rkt api service: dial tcp [::1]:15441: getsockopt: connection refused
I0729 17:24:27.662641 1 fs.go:116] Filesystem partitions: map[/dev/mapper/docker-202:1-263156-976049b5a1f76e87e6f26c7ae5ec7c50cf2887415332b91bcf951e021abc476c:{mountpoint:/rootfs/var/lib/docker/devicemapper/mnt/976049b5a1f76e87e6f26c7ae5ec7c50cf2887415332b91bcf951e021abc476c major:253 minor:2 fsType: blockSize:0} /dev/mapper/docker-202:1-263156-921422af56bbcbc069bfd69f64a66930b59aff1a8fe35d8a822c018b0c31ae0d:{mountpoint:/rootfs/var/lib/docker/devicemapper/mnt/921422af56bbcbc069bfd69f64a66930b59aff1a8fe35d8a822c018b0c31ae0d major:253 minor:8 fsType: blockSize:0} /dev/mapper/docker-202:1-263156-64ee72db5e151e1faca08c96b32742d02d9160a24259ff619054d2c7892d6fc5:{mountpoint:/rootfs/var/lib/docker/devicemapper/mnt/64ee72db5e151e1faca08c96b32742d02d9160a24259ff619054d2c7892d6fc5 major:253 minor:6 fsType: blockSize:0} /dev/mapper/docker-202:1-263156-51f2f202b2005306da8baec231817c12f1dec8287d688b6a59701918e5beb7c4:{mountpoint:/ major:253 minor:7 fsType:xfs blockSize:0} /dev/xvda1:{mountpoint:/var/lib/docker/devicemapper major:202 minor:1 fsType:ext4 blockSize:0} /dev/mapper/docker-202:1-263156-b9d30b4af429b43f5e72401d4c2a82865131e95704eb4baf25e361458068aaa5:{mountpoint:/rootfs/var/lib/docker/devicemapper/mnt/b9d30b4af429b43f5e72401d4c2a82865131e95704eb4baf25e361458068aaa5 major:253 minor:1 fsType: blockSize:0} /dev/mapper/docker-202:1-263156-7407c840a7d8f71970b91c00596c19cea8c39995c7fa15a5d5506680e082399e:{mountpoint:/rootfs/var/lib/docker/devicemapper/mnt/7407c840a7d8f71970b91c00596c19cea8c39995c7fa15a5d5506680e082399e major:253 minor:5 fsType: blockSize:0} /dev/mapper/docker-202:1-263156-f3ac70f398de460771c1e640923912f688292c9d4ed93f84d1d9885c7f6fcd8d:{mountpoint:/rootfs/var/lib/docker/devicemapper/mnt/f3ac70f398de460771c1e640923912f688292c9d4ed93f84d1d9885c7f6fcd8d major:253 minor:3 fsType: blockSize:0} /dev/mapper/docker-202:1-263156-67d7e7a0be08cbcf82ed8fc5c19b38985c18c3fe185ad254c6b578215c2a579b:{mountpoint:/rootfs/var/lib/docker/devicemapper/mnt/67d7e7a0be08cbcf82ed8fc5c19b38985c18c3fe185ad254c6b578215c2a579b major:253 minor:4 fsType: blockSize:0}]
I0729 17:24:27.668188 1 info.go:47] Couldn't collect info from any of the files in "/rootfs/etc/machine-id,/var/lib/dbus/machine-id"
I0729 17:24:27.668247 1 manager.go:184] Machine: {NumCores:4 CpuFrequency:2793350 MemoryCapacity:7846440960 MachineID: SystemUUID:EC20230A-BB42-DA0C-E8A7-9BF6D3FE0C65 BootID:97dafa01-8d3a-454e-86d2-c2d27e90e4f2 Filesystems:[{Device:/dev/mapper/docker-202:1-263156-7407c840a7d8f71970b91c00596c19cea8c39995c7fa15a5d5506680e082399e Capacity:10725883904 Type:vfs Inodes:10484736} {Device:/dev/mapper/docker-202:1-263156-f3ac70f398de460771c1e640923912f688292c9d4ed93f84d1d9885c7f6fcd8d Capacity:10725883904 Type:vfs Inodes:10484736} {Device:/dev/mapper/docker-202:1-263156-67d7e7a0be08cbcf82ed8fc5c19b38985c18c3fe185ad254c6b578215c2a579b Capacity:10725883904 Type:vfs Inodes:10484736} {Device:/dev/mapper/docker-202:1-263156-64ee72db5e151e1faca08c96b32742d02d9160a24259ff619054d2c7892d6fc5 Capacity:10725883904 Type:vfs Inodes:10484736} {Device:/dev/mapper/docker-202:1-263156-51f2f202b2005306da8baec231817c12f1dec8287d688b6a59701918e5beb7c4 Capacity:10725883904 Type:vfs Inodes:10484736} {Device:/dev/xvda1 Capacity:52708352000 Type:vfs Inodes:3276800} {Device:/dev/mapper/docker-202:1-263156-b9d30b4af429b43f5e72401d4c2a82865131e95704eb4baf25e361458068aaa5 Capacity:10725883904 Type:vfs Inodes:10484736} {Device:/dev/mapper/docker-202:1-263156-976049b5a1f76e87e6f26c7ae5ec7c50cf2887415332b91bcf951e021abc476c Capacity:10725883904 Type:vfs Inodes:10484736} {Device:/dev/mapper/docker-202:1-263156-921422af56bbcbc069bfd69f64a66930b59aff1a8fe35d8a822c018b0c31ae0d Capacity:10725883904 Type:vfs Inodes:10484736}] DiskMap:map[253:2:{Name:dm-2 Major:253 Minor:2 Size:10737418240 Scheduler:none} 253:3:{Name:dm-3 Major:253 Minor:3 Size:10737418240 Scheduler:none} 253:4:{Name:dm-4 Major:253 Minor:4 Size:10737418240 Scheduler:none} 253:5:{Name:dm-5 Major:253 Minor:5 Size:10737418240 Scheduler:none} 253:6:{Name:dm-6 Major:253 Minor:6 Size:10737418240 Scheduler:none} 253:7:{Name:dm-7 Major:253 Minor:7 Size:10737418240 Scheduler:none} 253:8:{Name:dm-8 Major:253 Minor:8 Size:10737418240 Scheduler:none} 253:1:{Name:dm-1 Major:253 Minor:1 Size:10737418240 Scheduler:none} 202:0:{Name:xvda Major:202 Minor:0 Size:53687091200 Scheduler:noop} 253:0:{Name:dm-0 Major:253 Minor:0 Size:107374182400 Scheduler:none}] NetworkDevices:[{Name:eth0 MacAddress:22:00:0b:19:02:86 Speed:0 Mtu:9001}] Topology:[{Id:0 Memory:7846440960 Cores:[{Id:0 Threads:[0 2] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:262144 Type:Unified Level:2}]} {Id:1 Threads:[1 3] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:262144 Type:Unified Level:2}]}] Caches:[{Size:26214400 Type:Unified Level:3}]}] CloudProvider:AWS InstanceType:c3.xlarge InstanceID:i-3a1c33bc}
I0729 17:24:27.669717 1 manager.go:190] Version: {KernelVersion:4.4.14-24.50.amzn1.x86_64 ContainerOsVersion:Alpine Linux v3.2 DockerVersion:1.11.2 CadvisorVersion:0.23.1 CadvisorRevision:6dd63f4}
I0729 17:24:27.778545 1 factory.go:189] Registering Docker factory
E0729 17:24:27.778580 1 manager.go:231] Registration of the rkt container factory failed: unable to communicate with Rkt api service: rkt: cannot tcp Dial rkt api service: dial tcp [::1]:15441: getsockopt: connection refused
I0729 17:24:27.778589 1 factory.go:53] Registering systemd factory
I0729 17:24:27.782064 1 factory.go:85] Registering Raw factory
I0729 17:24:27.782084 1 manager.go:1023] Started watching for new ooms in manager
W0729 17:24:27.785786 1 manager.go:263] Could not configure a source for OOM detection, disabling OOM events: unable to find any kernel log file available from our set: [/var/log/kern.log /var/log/messages /var/log/syslog]
I0729 17:24:27.796681 1 manager.go:276] Starting recovery of all containers
I0729 17:24:27.796744 1 manager.go:281] Recovery completed
F0729 17:24:27.796778 1 cadvisor.go:145] Failed to start container manager: inotify_add_watch /var/lib/docker/devicemapper/mnt/51f2f202b2005306da8baec231817c12f1dec8287d688b6a59701918e5beb7c4/rootfs/rootfs/var/lib/docker/devicemapper/mnt/51f2f202b2005306da8baec231817c12f1dec8287d688b6a59701918e5beb7c4/rootfs/sys/fs/cgroup/blkio: no such file or directory
docker inspect ecs-Monitoring-7-cadvisor-acbff3bae6b0a5803000
[
{
"Id": "411895cca1a8a50a51436bd7d760cd4ba890d3a209687341937f8051ea178038",
"Created": "2016-07-29T17:24:25.967251876Z",
"Path": "/usr/bin/cadvisor",
"Args": [
"-logtostderr"
],
"State": {
"Status": "exited",
"Running": false,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 0,
"ExitCode": 255,
"Error": "",
"StartedAt": "2016-07-29T17:24:27.494623637Z",
"FinishedAt": "2016-07-29T17:24:27.84698598Z"
},
"Image": "sha256:4bc3588563b107ed7f755ddbbdf09cccdb19243671d512810da3ed0fef6f7581",
"ResolvConfPath": "/var/lib/docker/containers/411895cca1a8a50a51436bd7d760cd4ba890d3a209687341937f8051ea178038/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/411895cca1a8a50a51436bd7d760cd4ba890d3a209687341937f8051ea178038/hostname",
"HostsPath": "/var/lib/docker/containers/411895cca1a8a50a51436bd7d760cd4ba890d3a209687341937f8051ea178038/hosts",
"LogPath": "/var/lib/docker/containers/411895cca1a8a50a51436bd7d760cd4ba890d3a209687341937f8051ea178038/411895cca1a8a50a51436bd7d760cd4ba890d3a209687341937f8051ea178038-json.log",
"Name": "/ecs-Monitoring-7-cadvisor-acbff3bae6b0a5803000",
"RestartCount": 0,
"Driver": "devicemapper",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "",
"ExecIDs": null,
"HostConfig": {
"Binds": [
"/var/lib/docker/:/var/lib/docker:ro",
"/sys:/sys:ro",
"/var/run:/var/run",
"/:/rootfs:ro"
],
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {
"max-size": "10m"
}
},
"NetworkMode": "default",
"PortBindings": {
"8080/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "8080"
}
]
},
"RestartPolicy": {
"Name": "",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"CapAdd": null,
"CapDrop": null,
"Dns": null,
"DnsOptions": null,
"DnsSearch": null,
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": false,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": null,
"StorageOpt": null,
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"ConsoleSize": [
0,
0
],
"Isolation": "",
"CpuShares": 2,
"Memory": 134217728,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": null,
"BlkioDeviceReadBps": null,
"BlkioDeviceWriteBps": null,
"BlkioDeviceReadIOps": null,
"BlkioDeviceWriteIOps": null,
"CpuPeriod": 0,
"CpuQuota": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": null,
"DiskQuota": 0,
"KernelMemory": 0,
"MemoryReservation": 0,
"MemorySwap": 268435456,
"MemorySwappiness": -1,
"OomKillDisable": false,
"PidsLimit": 0,
"Ulimits": null,
"CpuCount": 0,
"CpuPercent": 0,
"BlkioIOps": 0,
"BlkioBps": 0,
"SandboxSize": 0
},
"GraphDriver": {
"Name": "devicemapper",
"Data": {
"DeviceId": "298",
"DeviceName": "docker-202:1-263156-51f2f202b2005306da8baec231817c12f1dec8287d688b6a59701918e5beb7c4",
"DeviceSize": "10737418240"
}
},
"Mounts": [
{
"Source": "/",
"Destination": "/rootfs",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
},
{
"Source": "/var/lib/docker",
"Destination": "/var/lib/docker",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
},
{
"Source": "/sys",
"Destination": "/sys",
"Mode": "ro",
"RW": false,
"Propagation": "rprivate"
},
{
"Source": "/var/run",
"Destination": "/var/run",
"Mode": "",
"RW": true,
"Propagation": "rprivate"
}
],
"Config": {
"Hostname": "411895cca1a8",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"ExposedPorts": {
"8080/tcp": {}
},
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"GLIBC_VERSION=2.23-r1"
],
"Cmd": null,
"Image": "google/cadvisor",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": [
"/usr/bin/cadvisor",
"-logtostderr"
],
"OnBuild": null,
"Labels": {
"com.amazonaws.ecs.container-name": "cadvisor"
}
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "f200bda7f90b9f320a339c2afeb004e4e78ea2412cb9426eb67e3204e4d6ab77",
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"Ports": null,
"SandboxKey": "/var/run/docker/netns/f200bda7f90b",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"bridge": {
"IPAMConfig": null,
"Links": null,
"Aliases": null,
"NetworkID": "4ec7c39c680cc751a5c66372e8ccd647a3f9eec59c1f74066a37435f5e8c1eae",
"EndpointID": "",
"Gateway": "",
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": ""
}
}
}
}
]
docker info
Containers: 12
Running: 8
Paused: 0
Stopped: 4
Images: 25
Server Version: 1.11.2
Storage Driver: devicemapper
Pool Name: docker-202:1-263156-pool
Pool Blocksize: 65.54 kB
Base Device Size: 10.74 GB
Backing Filesystem: xfs
Data file: /dev/loop0
Metadata file: /dev/loop1
Data Space Used: 5.302 GB
Data Space Total: 107.4 GB
Data Space Available: 45.81 GB
Metadata Space Used: 13.06 MB
Metadata Space Total: 2.147 GB
Metadata Space Available: 2.134 GB
Udev Sync Supported: true
Deferred Removal Enabled: false
Deferred Deletion Enabled: false
Deferred Deleted Device Count: 0
Data loop file: /var/lib/docker/devicemapper/devicemapper/data
WARNING: Usage of loopback devices is strongly discouraged for production use. Either use `--storage-opt dm.thinpooldev` or use `--storage-opt dm.no_warn_on_loop_devices=true` to suppress this warning.
Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
Library Version: 1.02.93-RHEL7 (2015-01-28)
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: host bridge null
Kernel Version: 4.4.14-24.50.amzn1.x86_64
Operating System: Amazon Linux AMI 2016.03
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 7.308 GiB
Name:
ID:
Docker Root Dir: /var/lib/docker
Debug mode (client): false
Debug mode (server): false
Registry: https://index.docker.io/v1/
marksullivancrowd
All 10 comments
I am getting the same error as @marksullivancrowd
cajturner
on 4 Aug 2016
I am also getting same error
I0921 10:51:49.077889 1 storagedriver.go:50] Caching stats in memory for 2m0s
I0921 10:51:49.079061 1 manager.go:140] cAdvisor running in container: "/docker/f388d0af1e7af95910b978cf5952457ce33f75aa8338613f1db680be5d6c7c1c"
W0921 10:51:49.088118 1 manager.go:148] unable to connect to Rkt api service: rkt: cannot tcp Dial rkt api service: dial tcp [::1]:15441: getsockopt: connection refused
I0921 10:51:49.095053 1 fs.go:116] Filesystem partitions: map[/dev/mapper/docker-253:1-9144532-367762f1736f18be685e3b77748876f38ab6b65355943fadb060be71c8676c70:{mountpoint:/ major:252 minor:2 fsType:xfs blockSize:0} /dev/vda1:{mountpoint:/var/lib/docker/devicemapper major:253 minor:1 fsType:xfs blockSize:0} /dev/mapper/docker-253:1-9144532-c92bccf7a2ec59be6d94486cf7e58e346e305fe5f986ba89ae85e54b8443a2c9:{mountpoint:/rootfs/var/lib/docker/devicemapper/mnt/c92bccf7a2ec59be6d94486cf7e58e346e305fe5f986ba89ae85e54b8443a2c9 major:252 minor:1 fsType:xfs blockSize:0}]
I0921 10:51:49.097158 1 manager.go:195] Machine: {NumCores:2 CpuFrequency:2600000 MemoryCapacity:8004255744 MachineID:1d188a370466406fbb093d606a0478eb SystemUUID:5271A133-2134-45B0-96FB-0C234A2D4F43 BootID:3f999247-cedc-4483-988a-68113136f62e Filesystems:[{Device:/dev/mapper/docker-253:1-9144532-367762f1736f18be685e3b77748876f38ab6b65355943fadb060be71c8676c70 Capacity:10725883904 Type:vfs Inodes:10484736 HasInodes:true} {Device:/dev/vda1 Capacity:12873142272 Type:vfs Inodes:12581664 HasInodes:true} {Device:/dev/mapper/docker-253:1-9144532-c92bccf7a2ec59be6d94486cf7e58e346e305fe5f986ba89ae85e54b8443a2c9 Capacity:10725883904 Type:vfs Inodes:10484736 HasInodes:true}] DiskMap:map[252:0:{Name:dm-0 Major:252 Minor:0 Size:107374182400 Scheduler:none} 252:1:{Name:dm-1 Major:252 Minor:1 Size:10737418240 Scheduler:none} 252:2:{Name:dm-2 Major:252 Minor:2 Size:10737418240 Scheduler:none} 253:0:{Name:vda Major:253 Minor:0 Size:12884901888 Scheduler:none}] NetworkDevices:[{Name:eth0 MacAddress:fa:16:3e:51:21:a3 Speed:0 Mtu:1454}] Topology:[{Id:0 Memory:8388198400 Cores:[{Id:0 Threads:[0] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]} {Id:1 Memory:0 Cores:[{Id:0 Threads:[1] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
I0921 10:51:49.097712 1 manager.go:201] Version: {KernelVersion:3.10.0-327.10.1.el7.x86_64 ContainerOsVersion:Alpine Linux v3.4 DockerVersion:1.12.1 CadvisorVersion:v0.24.0 CadvisorRevision:0cdf491}
E0921 10:51:49.102474 1 factory.go:291] devicemapper filesystem stats will not be reported: unable to find thin_ls binary
I0921 10:51:49.102491 1 factory.go:295] Registering Docker factory
W0921 10:51:49.102518 1 manager.go:244] Registration of the rkt container factory failed: unable to communicate with Rkt api service: rkt: cannot tcp Dial rkt api service: dial tcp [::1]:15441: getsockopt: connection refused
I0921 10:51:49.102525 1 factory.go:54] Registering systemd factory
I0921 10:51:49.102657 1 factory.go:86] Registering Raw factory
I0921 10:51:49.102786 1 manager.go:1082] Started watching for new ooms in manager
W0921 10:51:49.103264 1 manager.go:272] Could not configure a source for OOM detection, disabling OOM events: unable to find any kernel log file available from our set: [/var/log/kern.log /var/log/messages /var/log/syslog]
I0921 10:51:49.103910 1 manager.go:285] Starting recovery of all containers
I0921 10:51:49.164319 1 manager.go:290] Recovery completed
F0921 10:51:49.164348 1 cadvisor.go:151] Failed to start container manager: inotify_add_watch /sys/fs/cgroup/cpuacct,cpu: no such file or directory
ripun
on 21 Sep 2016
Looks like the same error that is fixed by this pending PR when running
cAdvisor in a container.
https://github.com/google/cadvisor/pull/1474
On Wednesday, September 21, 2016, Ripunjay Godhani [email protected]
wrote:
I am also getting same error
I0921 10:51:49.077889 1 storagedriver.go:50] Caching stats in memory for
2m0s
I0921 10:51:49.079061 1 manager.go:140] cAdvisor running in container:
"/docker/f388d0af1e7af95910b978cf5952457ce33f75aa8338613f1db680be5d6c7c1c"
W0921 10:51:49.088118 1 manager.go:148] unable to connect to Rkt api
service: rkt: cannot tcp Dial rkt api service: dial tcp [::1]:15441:
getsockopt: connection refused
I0921 10:51:49.095053 1 fs.go:116] Filesystem partitions:
map[/dev/mapper/docker-253:1-9144532-367762f1736f18be685e3b77748876
f38ab6b65355943fadb060be71c8676c70:{mountpoint:/ major:252 minor:2
fsType:xfs blockSize:0} /dev/vda1:{mountpoint:/var/lib/docker/devicemapper
major:253 minor:1 fsType:xfs blockSize:0} /dev/mapper/docker-253:1-
9144532-c92bccf7a2ec59be6d94486cf7e58e346e305fe5f986ba89ae85e54b8443
a2c9:{mountpoint:/rootfs/var/lib/docker/devicemapper/mnt/
c92bccf7a2ec59be6d94486cf7e58e346e305fe5f986ba89ae85e54b8443a2c9
major:252 minor:1 fsType:xfs blockSize:0}]
I0921 10:51:49.097158 1 manager.go:195] Machine: {NumCores:2
CpuFrequency:2600000 MemoryCapacity:8004255744 MachineID:
1d188a370466406fbb093d606a0478eb SystemUUID:5271A133-2134-45B0-96FB-0C234A2D4F43
BootID:3f999247-cedc-4483-988a-68113136f62e Filesystems:[{Device:/dev/
mapper/docker-253:1-9144532-367762f1736f18be685e3b77748876
f38ab6b65355943fadb060be71c8676c70 Capacity:10725883904 Type:vfs
Inodes:10484736 HasInodes:true} {Device:/dev/vda1 Capacity:12873142272
Type:vfs Inodes:12581664 HasInodes:true} {Device:/dev/mapper/docker-
253:1-9144532-c92bccf7a2ec59be6d94486cf7e58e346e305fe5f986ba89ae85e54b8443a2c9
Capacity:10725883904 Type:vfs Inodes:10484736 HasInodes:true}]
DiskMap:map[252:0:{Name:dm-0 Major:252 Minor:0 Size:107374182400
Scheduler:none} 252:1:{Name:dm-1 Major:252 Minor:1 Size:10737418240
Scheduler:none} 252:2:{Name:dm-2 Major:252 Minor:2 Size:10737418240
Scheduler:none} 253:0:{Name:vda Major:253 Minor:0 Size:12884901888
Scheduler:none}] NetworkDevices:[{Name:eth0 MacAddress:fa:16:3e:51:21:a3
Speed:0 Mtu:1454}] Topology:[{Id:0 Memory:8388198400 Cores:[{Id:0
Threads:[0] Caches:[{Size:32768 Type:Data Level:1} {Size:32768
Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]}
{Id:1 Memory:0 Cores:[{Id:0 Threads:[1] Caches:[{Size:32768 Type:Data
Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified
Level:2}]}] Caches:[]}] CloudProvider:Unknown InstanceType:Unknown
InstanceID:None}
I0921 10:51:49.097712 1 manager.go:201] Version:
{KernelVersion:3.10.0-327.10.1.el7.x86_64 ContainerOsVersion:Alpine Linux
v3.4 DockerVersion:1.12.1 CadvisorVersion:v0.24.0 CadvisorRevision:0cdf491}
E0921 10:51:49.102474 1 factory.go:291] devicemapper filesystem stats will
not be reported: unable to find thin_ls binary
I0921 10:51:49.102491 1 factory.go:295] Registering Docker factory
W0921 10:51:49.102518 1 manager.go:244] Registration of the rkt container
factory failed: unable to communicate with Rkt api service: rkt: cannot tcp
Dial rkt api service: dial tcp [::1]:15441: getsockopt: connection refused
I0921 10:51:49.102525 1 factory.go:54] Registering systemd factory
I0921 10:51:49.102657 1 factory.go:86] Registering Raw factory
I0921 10:51:49.102786 1 manager.go:1082] Started watching for new ooms in
manager
W0921 10:51:49.103264 1 manager.go:272] Could not configure a source for
OOM detection, disabling OOM events: unable to find any kernel log file
available from our set: [/var/log/kern.log /var/log/messages
/var/log/syslog]
I0921 10:51:49.103910 1 manager.go:285] Starting recovery of all containers
I0921 10:51:49.164319 1 manager.go:290] Recovery completed
F0921 10:51:49.164348 1 cadvisor.go:151] Failed to start container
manager: inotify_add_watch /sys/fs/cgroup/cpuacct,cpu: no such file or
directory—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/google/cadvisor/issues/1401#issuecomment-248580919,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AF8dbFW0cD0PXOz5YTX_ZYDHL2i-X2j8ks5qsRD8gaJpZM4JYadx
.
derekwaynecarr
on 21 Sep 2016
Hi,
I don't think the above issue is fixed, I have tried both cadvisor and cadvisor-canary and am getting similar symptoms on an Amazon Linux instance...
I0208 22:08:06.528378 1 storagedriver.go:50] Caching stats in memory for 2m0s
I0208 22:08:06.528672 1 manager.go:140] cAdvisor running in container: "/docker/aeca51a748adbe6c12dc08550ed15024194544b4235fcb66144bee3dbac08334"
W0208 22:08:06.542050 1 manager.go:148] unable to connect to Rkt api service: rkt: cannot tcp Dial rkt api service: dial tcp [::1]:15441: getsockopt: connection refused
I0208 22:08:06.566575 1 fs.go:116] Filesystem partitions: map[/dev/mapper/docker-202:1-395786-98d5d321dc375464f3fb4cdc61dfc09e97c813ce39fe7bde32c1cdef15a7f7b7:{mountpoint:/ major:253 minor:13 fsType:xfs blockSize:0} /dev/xvda1:{mountpoint:/var/lib/docker/devicemapper major:202 minor:1 fsType:ext4 blockSize:0}]
I0208 22:08:06.570021 1 info.go:47] Couldn't collect info from any of the files in "/etc/machine-id,/var/lib/dbus/machine-id"
I0208 22:08:06.570107 1 manager.go:195] Machine: {NumCores:2 CpuFrequency:3000000 MemoryCapacity:16037875712 MachineID: SystemUUID:EC2FE94F-E16E-29BC-DC54-897C42293EDC BootID:e22f450d-be25-4472-a042-b290c9e76394 Filesystems:[{Device:/dev/mapper/docker-202:1-395786-98d5d321dc375464f3fb4cdc61dfc09e97c813ce39fe7bde32c1cdef15a7f7b7 Capacity:10725883904 Type:vfs Inodes:10484736 HasInodes:true} {Device:/dev/xvda1 Capacity:8318783488 Type:vfs Inodes:524288 HasInodes:true}] DiskMap:map[253:0:{Name:dm-0 Major:253 Minor:0 Size:107374182400 Scheduler:none} 253:3:{Name:dm-3 Major:253 Minor:3 Size:10737418240 Scheduler:none} 253:7:{Name:dm-7 Major:253 Minor:7 Size:10737418240 Scheduler:none} 253:8:{Name:dm-8 Major:253 Minor:8 Size:10737418240 Scheduler:none} 253:9:{Name:dm-9 Major:253 Minor:9 Size:10737418240 Scheduler:none} 202:0:{Name:xvda Major:202 Minor:0 Size:8589934592 Scheduler:noop} 253:10:{Name:dm-10 Major:253 Minor:10 Size:10737418240 Scheduler:none} 253:6:{Name:dm-6 Major:253 Minor:6 Size:10737418240 Scheduler:none} 253:12:{Name:dm-12 Major:253 Minor:12 Size:10737418240 Scheduler:none} 253:4:{Name:dm-4 Major:253 Minor:4 Size:10737418240 Scheduler:none} 253:1:{Name:dm-1 Major:253 Minor:1 Size:10737418240 Scheduler:none} 253:11:{Name:dm-11 Major:253 Minor:11 Size:10737418240 Scheduler:none} 253:13:{Name:dm-13 Major:253 Minor:13 Size:10737418240 Scheduler:none} 253:2:{Name:dm-2 Major:253 Minor:2 Size:10737418240 Scheduler:none} 253:5:{Name:dm-5 Major:253 Minor:5 Size:10737418240 Scheduler:none}] NetworkDevices:[{Name:eth0 MacAddress:12:18:27:50:94:b8 Speed:0 Mtu:9001}] Topology:[{Id:0 Memory:16037875712 Cores:[{Id:0 Threads:[0 1] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:262144 Type:Unified Level:2}]}] Caches:[{Size:47185920 Type:Unified Level:3}]}] CloudProvider:AWS InstanceType:r4.large InstanceID:i-0176c03a9ed8c4cb3}
I0208 22:08:06.570741 1 manager.go:201] Version: {KernelVersion:4.4.41-36.55.amzn1.x86_64 ContainerOsVersion:Alpine Linux v3.4 DockerVersion:1.12.6 CadvisorVersion:v0.24.1 CadvisorRevision:ae6934c}
E0208 22:08:06.580310 1 factory.go:291] devicemapper filesystem stats will not be reported: unable to find thin_ls binary
I0208 22:08:06.580328 1 factory.go:295] Registering Docker factory
W0208 22:08:06.580342 1 manager.go:244] Registration of the rkt container factory failed: unable to communicate with Rkt api service: rkt: cannot tcp Dial rkt api service: dial tcp [::1]:15441: getsockopt: connection refused
I0208 22:08:06.580349 1 factory.go:54] Registering systemd factory
I0208 22:08:06.580781 1 factory.go:86] Registering Raw factory
I0208 22:08:06.581134 1 manager.go:1082] Started watching for new ooms in manager
W0208 22:08:06.581430 1 manager.go:272] Could not configure a source for OOM detection, disabling OOM events: unable to find any kernel log file available from our set: [/var/log/kern.log /var/log/messages /var/log/syslog]
I0208 22:08:06.581769 1 manager.go:285] Starting recovery of all containers
I0208 22:08:06.581837 1 manager.go:290] Recovery completed
F0208 22:08:06.581865 1 cadvisor.go:151] Failed to start container manager: inotify_add_watch /var/lib/docker/devicemapper/mnt/98d5d321dc375464f3fb4cdc61dfc09e97c813ce39fe7bde32c1cdef15a7f7b7/rootfs/sys/fs/cgroup/cpu: no such file or directory
bmouthrob
on 9 Feb 2017
For those who trapped into that issue... "latest" is kinda outdated currently, tag is 5 month old. v0.25.0 works fine for me with Amazon ECS optimized AMI.
dene14
on 24 Mar 2017
I'll bump latest (@dashpole FYI)
timstclair
on 24 Mar 2017
Hi!
Hitting the same issue. I've tried with v0.25.0 to no avail! Anyone had any joy in getting this work on ECS optimized AMI?
Couldn't collect info from any of the files in "/rootfs/etc/machine-id,/var/lib/dbus/machine-id"
Damien
damienmarshall
on 19 Oct 2017
Whoops, sorry, meant to tag @timstclair - does v0.25.0 work on latest ECS optimized AMI for you?
Thanks!
Damien
damienmarshall
on 20 Oct 2017
❓ has there been a resolution to this issue?
ronmessana-concerto
on 20 Feb 2019
cAdvisor's latest tag is now v0.32.0.
It looks like the original issue was fixed a long time ago.
The Couldn't collect info from any of the files in "/rootfs/etc/machine-id,/var/lib/dbus/machine-id" is a different issue. It is likely because the mounts for cAdvisor aren't specified correctly.
dashpole
on 20 Feb 2019
Related issues
octete
·
6Comments
ghost
·
5Comments
dashpole
·
7Comments
cheneypan
·
4Comments
NichUK
·
5Comments
Most helpful comment
For those who trapped into that issue... "latest" is kinda outdated currently, tag is 5 month old. v0.25.0 works fine for me with Amazon ECS optimized AMI.