Node_exporter: Node exporter systemd error

[mg03]

I am tryting to get systemd metrics keep getting following error

"level":"error","msg":"ERROR: systemd collector failed after 0.000167s: couldn't get units states: couldn't get dbus connection: dial unix /var/run/dbus/system_bus_socket: connect: no such file or directory","source":"node_exporter.go:91","time":"2016-08-04T02:34:13Z"}

I ran node_exporter as root but still get same error

[grobie]

@mg03 Have you been able to fix the problem? It seems your systemd setup opens the socket at a different location.

[mg03]

Hi @grobie

systemctl list-units | grep dbus
dbus.service                                                                                         loaded active running   D-Bus System Message Bus
dbus.socket                                                                                          loaded active running   D-Bus System Message Bus Socket

systemctl cat dbus.service
# /lib/systemd/system/dbus.service
[Unit]
Description=D-Bus System Message Bus
Documentation=man:dbus-daemon(1)
Requires=dbus.socket
# we don't properly stop D-Bus (see ExecStop=), thus disallow restart
RefuseManualStart=yes

[Service]
ExecStart=/usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation
ExecReload=/usr/bin/dbus-send --print-reply --system --type=method_call --dest=org.freedesktop.DBus / org.freedesktop.DBus.ReloadConfig
ExecStop=/bin/true
KillMode=none
OOMScoreAdjust=-900

systemctl cat dbus.socket
# /lib/systemd/system/dbus.socket
[Unit]
Description=D-Bus System Message Bus Socket

[Socket]
ListenStream=/var/run/dbus/system_bus_socket

ls -ltr /var/run/dbus/system_bus_socket
srw-rw-rw- 1 root root 0 Aug 12 06:34 /var/run/dbus/system_bus_socket

I see its listening on /var/run/dbus/system_bus_socket

Does node_exporter need specific privileges/capabilities?

[grobie]

It works without issues for me. How do you run the node_exporter?

[victorgp]

I'm having the same issue using CoreOS and this can be fixed by mounting /var/run/dbus/system_bus_socket as a volume

[sotoz]

I'm having the same problem with debian8.

[sudermanjr]

I also have this issue on CoreOS. I tried mounting volume mounting to the container, but that didn't help. @victorgp Can you post the exact mount you used to fix?

Thanks!

[sotoz]

The problem in my case was that dbus was missing from the linux installation. I solved it by doing a simple apt-get install dbus :)

[victorgp]

I'm running the container in a Kubernetes Pod and mounting the volumes doing nothing special, the equivalent docker command to run the container would be:

-v /var/run/dbus/system_bus_socket:/var/run/dbus/system_bus_socket

Same path in origin and host

Or with kubernetes:

        volumeMounts:
        - mountPath: /var/run/dbus/system_bus_socket
          name: system-dbus-socket
          readOnly: true

and

      volumes:
      - hostPath:
          path: /var/run/dbus/system_bus_socket
        name: system-dbus-socket

[sudermanjr]

@victorgp Thanks! I am using fleet and the

-v /var/run/dbus/system_bus_socket:/var/run/dbus/system_bus_socket

worked great. That is what I had tried before, but I had a typo that I missed. Works like a charm now.

[SuperQ]

Looks like this is solved.

[i0pr1m3]

Looks like I have the same issue on CentOS 7.6.

nodeexporter | time="2019-03-17T18:08:14Z" level=error msg="ERROR: systemd collector failed after 0.000149s: couldn't get units: couldn't get dbus connection: dial unix /var/run/dbus/system_bus_socket: connect: no such file or directory" source="collector.go:132"

When I mount a volume I see:

nodeexporter | time="2019-03-17T18:16:24Z" level=error msg="ERROR: systemd collector failed after 0.002088s: couldn't get units: couldn't get dbus connection: dbus: authentication failed" source="collector.go:132"

  nodeexporter:
    image: prom/node-exporter
    container_name: nodeexporter
    volumes:
      - /proc:/host/proc:ro
      - /sys:/host/sys:ro
      - /:/rootfs:ro
      - /run/dbus/system_bus_socket:/var/run/dbus/system_bus_socket:ro
    command:
      - '--path.procfs=/host/proc'
      - '--path.rootfs=/rootfs'
      - '--path.sysfs=/host/sys'
      - '--collector.filesystem.ignored-mount-points=^/(sys|proc|dev|host|etc)($$|/)'
      - '--collector.systemd'

# /usr/lib/systemd/system/dbus.socket
[Unit]
Description=D-Bus System Message Bus Socket

[Socket]
ListenStream=/run/dbus/system_bus_socket

Do you have any ideas?

[skpenpen]

Solution :
user: root

[Ninjamannn]

Solution for my case in docker-compose:

---

nodeexporter:
network_mode: host
image: prom/node-exporter:latest
container_name: nodeexporter
user: root
privileged: true
volumes:
- /proc:/host/proc:ro
- /sys:/host/sys:ro
- /:/rootfs:ro
- /run/systemd/private:/run/systemd/private:ro
command:
- '--path.procfs=/host/proc'
- '--path.sysfs=/host/sys'
- '--collector.systemd'
- '--collector.filesystem.ignored-mount-points=^/(sys|proc|dev|host|etc)($$|/)'
# restart: unless-stopped
expose:
- 9100

---