Node_exporter: Node exporter systemd error related to dbus

[i0pr1m3]

Host operating system: output of uname -a

Linux 3.10.0-957.5.1.el7.x86_64 #1 SMP Fri Feb 1 14:54:57 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

node_exporter version: output of node_exporter --version

node_exporter, version 0.17.0 (branch: HEAD, revision: f6f6194a436b9a63d0439abc585c76b19a206b21

node_exporter command line flags

--collector.systemd

Are you running node_exporter in Docker?

Yes

What did you do that produced an error?

docker-compose up

  nodeexporter:
    image: prom/node-exporter
    container_name: nodeexporter
    volumes:
      - /proc:/host/proc:ro
      - /sys:/host/sys:ro
      - /:/rootfs:ro
      - /run/dbus/system_bus_socket:/var/run/dbus/system_bus_socket:ro
    command:
      - '--path.procfs=/host/proc'
      - '--path.rootfs=/rootfs'
      - '--path.sysfs=/host/sys'
      - '--collector.filesystem.ignored-mount-points=^/(sys|proc|dev|host|etc)($$|/)'
      - '--collector.systemd'

# /usr/lib/systemd/system/dbus.socket
[Unit]
Description=D-Bus System Message Bus Socket

[Socket]
ListenStream=/run/dbus/system_bus_socket

What did you expect to see?

systemd metrics in prometheus

What did you see instead?

Looks like I have the same issue on CentOS 7.6 as well as in #273

nodeexporter | time="2019-03-17T18:08:14Z" level=error msg="ERROR: systemd collector failed after 0.000149s: couldn't get units: couldn't get dbus connection: dial unix /var/run/dbus/system_bus_socket: connect: no such file or directory" source="collector.go:132"

When I mount a volume (/run/dbus/system_bus_socket:/var/run/dbus/system_bus_socket:ro) I see:
nodeexporter | time="2019-03-17T18:16:24Z" level=error msg="ERROR: systemd collector failed after 0.002088s: couldn't get units: couldn't get dbus connection: dbus: authentication failed" source="collector.go:132"

[SuperQ]

Have you tried making the systemd socket :rw?

[i0pr1m3]

@SuperQ
Yes, I've tried.

[aixeshunter]

Try run as root.

[i0pr1m3]

Try run as root.

I've tried that as well.

[cnorthwood]

I get this when I find the UID that node_exporter runs as inside the Docker container doesn't correspond to a user on the host. sudo useradd prom-nobody -d / -g 99 -s /sbin/nologin -u 65534 works for me to fix this

[CodyGuo]

docker couldn't get dbus connection: dbus: authentication failed
the nobody user ID in the container is 65534, an error will occur if there is no user ID of 65534 on the host. you must create a user with user ID of 65534 to solve this problem.
sudo useradd -s /bin/false -u 65534 node-exporter