Cadvisor: v32.0 - Failed to get system UUID: open /etc/machine-id: no such file or directory (Centos 7.6)

[BootsSiR]

OS: CentOS Linux release 7.6.1810
_SELinux is active_

docker-compose.yml:

cadvisor:
    container_name: cadvisor
    image: google/cadvisor:v0.32.0
    volumes:
      - /:/rootfs:ro
      - /var/run:/var/run:rw
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro
      - /cgroup:/cgroup:ro 
    ports:
      - '9090:8080'
    privileged: true      
    restart: always

If I use v32.0, I get this output and nothing works

cadvisor | E0129 13:31:43.708149 1 info.go:140] Failed to get system UUID: open /etc/machine-id: no such file or directory

If I use v31.0, I get this output and it appears things work fine (I can access the webui)

cadvisor | I0129 13:33:06.507201 1 storagedriver.go:50] Caching stats in memory for 2m0s
cadvisor | I0129 13:33:06.507728 1 manager.go:155] cAdvisor running in container: "/sys/fs/cgroup/cpu,cpuacct"
cadvisor | I0129 13:33:06.570169 1 fs.go:142] Filesystem UUIDs: map[]
cadvisor | I0129 13:33:06.570199 1 fs.go:143] Filesystem partitions: map[tmpfs:{mountpoint:/dev major:0 minor:44 fsType:tmpfs blockSize:0} /dev/sda:{mountpoint:/var/lib/docker major:8 minor:0 fsType:ext4 blockSize:0} shm:{mountpoint:/rootfs/var/lib/docker/containers/ac8674881bf8a67cb74d2726661194d449924a5cd8de2fb2136044d8991d1fb6/mounts/shm major:0 minor:41 fsType:tmpfs blockSize:0}]
cadvisor | E0129 13:33:06.579683 1 info.go:140] Failed to get system UUID: open /etc/machine-id: no such file or directory
cadvisor | I0129 13:33:06.580167 1 manager.go:229] Machine: {NumCores:4 CpuFrequency:1999998 MemoryCapacity:8201080832 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:cb018d6767ca4c8983df25647a8794b0 SystemUUID: BootID:1bc629cd-15e6-4cd5-9905-70c78f61b18b Filesystems:[{Device:tmpfs DeviceMajor:0 DeviceMinor:44 Capacity:67108864 Type:vfs Inodes:1001108 HasInodes:true} {Device:/dev/sda DeviceMajor:8 DeviceMinor:0 Capacity:167017951232 Type:vfs Inodes:10112000 HasInodes:true} {Device:shm DeviceMajor:0 DeviceMinor:41 Capacity:67108864 Type:vfs Inodes:1001108 HasInodes:true} {Device:overlay DeviceMajor:0 DeviceMinor:40 Capacity:167017951232 Type:vfs Inodes:10112000 HasInodes:true}] DiskMap:map[8:0:{Name:sda Major:8 Minor:0 Size:169651208192 Scheduler:deadline} 8:16:{Name:sdb Major:8 Minor:16 Size:2147483648 Scheduler:deadline}] NetworkDevices:[{Name:br-01c2cf4aa990 MacAddress:02:42:d2:d3:76:f9 Speed:0 Mtu:1500} {Name:br-0c1136f830e3 MacAddress:02:42:52:7b:50:65 Speed:0 Mtu:1500} {Name:br-20cdb806ba71 MacAddress:02:42:a0:7d:33:79 Speed:0 Mtu:1500} {Name:br-3f49ee7b6836 MacAddress:02:42:4e:26:75:41 Speed:0 Mtu:1500} {Name:br-4e992eafd433 MacAddress:02:42:3e:f1:e9:dc Speed:0 Mtu:1500} {Name:br-593f605af16c MacAddress:02:42:de:06:4f:15 Speed:0 Mtu:1500} {Name:br-5b1c844a65bb MacAddress:02:42:70:96:0e:78 Speed:0 Mtu:1500} {Name:br-5ba64de99d49 MacAddress:02:42:5d:c4:17:38 Speed:0 Mtu:1500} {Name:br-5e907e3b6f6b MacAddress:02:42:14:cc:48:a1 Speed:0 Mtu:1500} {Name:br-80c701a6c840 MacAddress:02:42:b5:aa:48:6a Speed:0 Mtu:1500} {Name:br-b6f4c4e79414 MacAddress:02:42:89:90:14:3e Speed:0 Mtu:1500} {Name:br-c9247e85b5f7 MacAddress:02:42:15:c9:53:4f Speed:0 Mtu:1500} {Name:br-e02938a526b2 MacAddress:02:42:7f:e4:bb:df Speed:0 Mtu:1500} {Name:eth0 MacAddress:f2:3c:91:c5:b2:7b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:8589393920 Cores:[{Id:0 Threads:[0] Caches:[{Size:65536 Type:Data Level:1} {Size:65536 Type:Instruction Level:1} {Size:524288 Type:Unified Level:2}]}] Caches:[{Size:16777216 Type:Unified Level:3}]} {Id:1 Memory:0 Cores:[{Id:0 Threads:[1] Caches:[{Size:65536 Type:Data Level:1} {Size:65536 Type:Instruction Level:1} {Size:524288 Type:Unified Level:2}]}] Caches:[{Size:16777216 Type:Unified Level:3}]} {Id:2 Memory:0 Cores:[{Id:0 Threads:[2] Caches:[{Size:65536 Type:Data Level:1} {Size:65536 Type:Instruction Level:1} {Size:524288 Type:Unified Level:2}]}] Caches:[{Size:16777216 Type:Unified Level:3}]} {Id:3 Memory:0 Cores:[{Id:0 Threads:[3] Caches:[{Size:65536 Type:Data Level:1} {Size:65536 Type:Instruction Level:1} {Size:524288 Type:Unified Level:2}]}] Caches:[{Size:16777216 Type:Unified Level:3}]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
cadvisor | I0129 13:33:06.582277 1 manager.go:235] Version: {KernelVersion:3.10.0-957.1.3.el7.x86_64 ContainerOsVersion:Alpine Linux v3.7 DockerVersion:18.09.1 DockerAPIVersion:1.39 CadvisorVersion:v0.31.0 CadvisorRevision:fc17731a}
cadvisor | I0129 13:33:06.608589 1 factory.go:356] Registering Docker factory
cadvisor | I0129 13:33:06.611152 1 factory.go:136] Registering containerd factory
cadvisor | I0129 13:33:06.612419 1 factory.go:138] Registering mesos factory
cadvisor | I0129 13:33:06.612444 1 factory.go:54] Registering systemd factory
cadvisor | I0129 13:33:06.613453 1 factory.go:97] Registering Raw factory
cadvisor | I0129 13:33:06.614401 1 manager.go:1222] Started watching for new ooms in manager
cadvisor | I0129 13:33:06.618655 1 manager.go:365] Starting recovery of all containers
cadvisor | I0129 13:33:06.701418 1 manager.go:370] Recovery completed
cadvisor | I0129 13:33:06.792173 1 cadvisor.go:174] Starting cAdvisor version: v0.31.0-fc17731a on port 8080

The error still exists in v31, it just doesn't seem to be a show stopper. Any suggestions?

[dashpole]

Can you be more specific when you say "nothing works"? Does the kubelet simply log that and then exit, or does it hang somewhere, or does it just never report any container metrics?

[BootsSiR]

@dashpole I get that single line of logging and it's DOA until I stop the container. I cannot access the webui at all.

[dashpole]

I have looked through the changes between those versions, but can't see anything immediately that would affect this. This must be hanging in the machine.Info( function because we log the result immediately after that is called during initialization.

The only thing that happens between then is reading the machine id (/etc/machine-id) and the boot id (/proc/sys/kernel/random/boot_id), and getting all of the cloud info from the cloud-provider.

[hthiers]

Same problem here with Centos 7.6: info.go:140] Failed to get system UUID: open /etc/machine-id: no such file or directory
No web UI.

[pedropinheiro75]

Same problem using the compiled binary directly on a Amazon Linux AMI 2018.03:

4.14.123-86.109.amzn1.x86_64 #1 SMP x86_64 x86_64 x86_64 GNU/Linux

info.go:140] Failed to get system UUID: open /etc/machine-id: no such file or directory

[XSHui]

v0.33.0 Same problem.

[wc-matteo]

same on ubuntu 18.04

[mikzuit]

v0.34,0 and v0.36.0 same problem. I've tried v0.34.0 and v0.36.0 under Alpine 3.10 in docker container under Windows 10 with same info.go:140] Failed to get system UUID: open /etc/machine-id: no such file or directory and info.go:102] Failed to get system UUID: open /etc/machine-id: no such file or directory respectively , info message is every 5 minutes.

[calbot]

I'm getting this too. It just started out of the blue...

E1017 04:30:30.421766       1 info.go:140] Failed to get system UUID: open /etc/machine-id: no such file or directory
W1017 04:30:30.492874       1 manager.go:256] Could not configure a source for OOM detection, disabling OOM events: open /dev/kmsg: no such file or directory