Supervisor: CRITICAL (MainThread) [supervisor.core] System running in a unhealthy state and need manual intervention!

[georaspi]

Describe the issue

I have noticed that only today I got a constant warning error message in my Supervisor status:
"Your installtion is running in an unhealthy state."

I followed advice on whether the dbus daemon is running and it is actually running as it should.

I have tried to reboot the system, reload the supervisor, etc and still no solution.

Enviroment details

ha supervisor info
arch: armv7
channel: stable
debug: false
debug_block: false
diagnostics: null
healthy: false
ip_address: 172.30.32.2
logging: info
supported: false
timezone: Europe/London
update_available: false
version: 2020.12.2
version_latest: 2020.12.2
wait_boot: 5

Home Assistant Core Integration

version: 0.118.3
installation_type: Home Assistant Supervised
dev: false
hassio: true
docker: true
virtualenv: false
python_version: 3.8.6
os_name: Linux
os_version: 4.19.66-v7+
arch: armv7l
timezone: Europe/London

Home Assistant Cloud

logged_in: false
can_reach_cert_server: ok
can_reach_cloud_auth: ok
can_reach_cloud: ok

Hass.io

host_os: Raspbian GNU/Linux 9 (stretch)
update_channel: stable
supervisor_version: 2020.12.2
docker_version: 19.03.14
disk_total: 109.5 GB
disk_used: 16.4 GB
healthy: failed to load: Unhealthy
supported: failed to load: Unsupported
supervisor_api: ok
version_api: ok
installed_addons: Duck DNS (1.12.4), NGINX Home Assistant SSL proxy (3.0.1), File editor (5.2.0), Mosquitto broker (5.1), AdGuard Home (2.6.1), Glances (0.9.1), TasmoAdmin (0.13.1), WireGuard (0.4.0), Plex Media Server (2.4.1), Alsa & PulseAudio Fix (3.4), ESPHome (1.15.3)

Lovelace

dashboards: 1
mode: yaml
views: 11
resources: 11

Supervisor logs

20-12-03 17:40:36 INFO (MainThread) [supervisor.updater] Fetching update data from https://version.home-assistant.io/stable.json,
20-12-03 17:40:37 INFO (SyncWorker_0) [supervisor.docker.interface] Attaching to homeassistant/raspberrypi3-homeassistant with version 0.118.3,
20-12-03 17:40:37 INFO (MainThread) [supervisor.homeassistant.secrets] Loaded 1 Home Assistant secrets,
20-12-03 17:40:37 INFO (MainThread) [supervisor.hassos] No Home Assistant Operating System found,
20-12-03 17:40:38 INFO (MainThread) [supervisor.store.git] Loading add-on /data/addons/core repository,
20-12-03 17:40:38 INFO (MainThread) [supervisor.store.git] Loading add-on /data/addons/git/a0d7b954 repository,
20-12-03 17:40:38 INFO (MainThread) [supervisor.store.git] Loading add-on /data/addons/git/d4868bb1 repository,
20-12-03 17:40:39 INFO (MainThread) [supervisor.store] Loading add-ons from store: 67 all - 67 new - 0 remove,
20-12-03 17:40:39 INFO (MainThread) [supervisor.addons] Found 11 installed add-ons,
20-12-03 17:40:39 INFO (SyncWorker_2) [supervisor.docker.interface] Attaching to hassioaddons/sonweb-armv7 with version 0.13.1,
20-12-03 17:40:39 INFO (SyncWorker_0) [supervisor.docker.interface] Attaching to d4868bb1/armv7-addon-pulseaudio_fix with version 3.4,
20-12-03 17:40:39 INFO (SyncWorker_4) [supervisor.docker.interface] Attaching to hassioaddons/wireguard-armv7 with version 0.4.0,
20-12-03 17:40:39 INFO (SyncWorker_1) [supervisor.docker.interface] Attaching to hassioaddons/plex-armv7 with version 2.4.1,
20-12-03 17:40:39 INFO (SyncWorker_6) [supervisor.docker.interface] Attaching to homeassistant/armv7-addon-nginx_proxy with version 3.0.1,
20-12-03 17:40:39 INFO (SyncWorker_2) [supervisor.docker.interface] Attaching to hassioaddons/glances-armv7 with version 0.9.1,
20-12-03 17:40:39 INFO (SyncWorker_3) [supervisor.docker.interface] Attaching to esphome/esphome-hassio-armv7 with version 1.15.3,
20-12-03 17:40:39 INFO (SyncWorker_7) [supervisor.docker.interface] Attaching to homeassistant/armv7-addon-duckdns with version 1.12.4,
20-12-03 17:40:39 INFO (SyncWorker_5) [supervisor.docker.interface] Attaching to homeassistant/armv7-addon-configurator with version 5.2.0,
20-12-03 17:40:39 INFO (SyncWorker_4) [supervisor.docker.interface] Attaching to homeassistant/armv7-addon-mosquitto with version 5.1,
20-12-03 17:40:39 INFO (SyncWorker_0) [supervisor.docker.interface] Attaching to hassioaddons/adguard-armv7 with version 2.6.1,
20-12-03 17:40:40 INFO (MainThread) [supervisor.snapshots] Found 3 snapshot files,
20-12-03 17:40:40 INFO (MainThread) [supervisor.discovery] Loaded 2 messages,
20-12-03 17:40:40 INFO (MainThread) [supervisor.ingress] Loaded 5 ingress sessions,
20-12-03 17:40:40 INFO (MainThread) [supervisor.resolution.check] Starting system checks with state CoreState.SETUP,
20-12-03 17:40:40 INFO (MainThread) [supervisor.resolution.check] System checks complete,
20-12-03 17:40:40 INFO (MainThread) [supervisor.resolution.evaluate] Starting system evaluation with state CoreState.SETUP,
20-12-03 17:40:40 WARNING (MainThread) [supervisor.resolution.evaluations.base] Detected unsupported OS: Raspbian GNU/Linux 9 (stretch) (more-info: https://www.home-assistant.io/more-info/unsupported/os),
20-12-03 17:40:40 INFO (MainThread) [supervisor.resolution.evaluate] System evaluation complete,
20-12-03 17:40:40 INFO (MainThread) [__main__] Running Supervisor,
20-12-03 17:40:40 WARNING (MainThread) [supervisor.core] System running in a unsupported environment!,
20-12-03 17:40:40 CRITICAL (MainThread) [supervisor.core] System running in a unhealthy state and need manual intervention!,
20-12-03 17:40:40 INFO (MainThread) [supervisor.addons] Phase 'AddonStartup.INITIALIZE' starting 0 add-ons,
20-12-03 17:40:40 INFO (MainThread) [supervisor.addons] Phase 'AddonStartup.SYSTEM' starting 2 add-ons,
20-12-03 17:40:40 INFO (SyncWorker_7) [supervisor.docker.interface] Cleaning addon_core_mosquitto application,
20-12-03 17:40:43 INFO (SyncWorker_7) [supervisor.docker.addon] Starting Docker add-on homeassistant/armv7-addon-mosquitto with version 5.1,
20-12-03 17:40:44 INFO (SyncWorker_3) [supervisor.docker.interface] Cleaning addon_a0d7b954_sonweb application,
20-12-03 17:40:47 INFO (SyncWorker_3) [supervisor.docker.addon] Starting Docker add-on hassioaddons/sonweb-armv7 with version 0.13.1,
20-12-03 17:40:51 INFO (MainThread) [supervisor.services.modules.mqtt] Set core_mosquitto as service provider for mqtt,
20-12-03 17:40:52 INFO (MainThread) [supervisor.addons] Phase 'AddonStartup.SERVICES' starting 2 add-ons,
20-12-03 17:40:52 INFO (SyncWorker_5) [supervisor.docker.interface] Cleaning addon_core_duckdns application,
20-12-03 17:40:55 INFO (SyncWorker_5) [supervisor.docker.addon] Starting Docker add-on homeassistant/armv7-addon-duckdns with version 1.12.4,
20-12-03 17:40:56 WARNING (SyncWorker_6) [supervisor.docker.addon] Glances running with disabled protected mode!,
20-12-03 17:40:56 INFO (SyncWorker_6) [supervisor.docker.interface] Cleaning addon_a0d7b954_glances application,
20-12-03 17:40:58 INFO (SyncWorker_6) [supervisor.docker.addon] Starting Docker add-on hassioaddons/glances-armv7 with version 0.9.1,
20-12-03 17:41:01 INFO (MainThread) [supervisor.api.security] /host/info access from a0d7b954_sonweb,
20-12-03 17:41:03 INFO (MainThread) [supervisor.core] Start Home Assistant Core,
20-12-03 17:41:03 INFO (SyncWorker_3) [supervisor.docker.interface] Starting homeassistant,
20-12-03 17:41:03 INFO (MainThread) [supervisor.homeassistant.core] Wait until Home Assistant is ready,
20-12-03 17:41:04 INFO (MainThread) [supervisor.api.security] /host/info access from a0d7b954_glances,
20-12-03 17:41:13 INFO (MainThread) [supervisor.api.security] /dns/info access from a0d7b954_glances,
20-12-03 17:41:39 INFO (MainThread) [supervisor.homeassistant.api] Updated Home Assistant API token,
20-12-03 17:43:44 INFO (MainThread) [supervisor.homeassistant.core] Detect a running Home Assistant instance,
20-12-03 17:43:44 INFO (MainThread) [supervisor.addons] Phase 'AddonStartup.APPLICATION' starting 4 add-ons,
20-12-03 17:43:44 INFO (SyncWorker_7) [supervisor.docker.interface] Cleaning addon_core_nginx_proxy application,
20-12-03 17:43:50 INFO (SyncWorker_7) [supervisor.docker.addon] Starting Docker add-on homeassistant/armv7-addon-nginx_proxy with version 3.0.1,
20-12-03 17:43:51 INFO (SyncWorker_2) [supervisor.docker.interface] Cleaning addon_core_configurator application,
20-12-03 17:43:54 INFO (SyncWorker_2) [supervisor.docker.addon] Starting Docker add-on homeassistant/armv7-addon-configurator with version 5.2.0,
20-12-03 17:43:54 INFO (SyncWorker_6) [supervisor.docker.interface] Cleaning addon_a0d7b954_wireguard application,
20-12-03 17:43:57 INFO (SyncWorker_6) [supervisor.docker.addon] Starting Docker add-on hassioaddons/wireguard-armv7 with version 0.4.0,
20-12-03 17:43:57 INFO (SyncWorker_0) [supervisor.docker.interface] Cleaning addon_d4868bb1_pulseaudio_fix application,
20-12-03 17:44:01 INFO (SyncWorker_0) [supervisor.docker.addon] Starting Docker add-on d4868bb1/armv7-addon-pulseaudio_fix with version 3.4,
20-12-03 17:44:03 INFO (MainThread) [supervisor.api.security] /host/info access from a0d7b954_wireguard,
20-12-03 17:44:06 INFO (MainThread) [supervisor.misc.tasks] All core tasks are scheduled,
20-12-03 17:44:06 INFO (MainThread) [supervisor.misc.hwmon] Started Supervisor hardware monitor,
20-12-03 17:44:06 INFO (MainThread) [supervisor.core] Supervisor is up and running,
20-12-03 17:44:06 INFO (MainThread) [supervisor.host.info] Updating local host information,
20-12-03 17:44:06 INFO (MainThread) [supervisor.updater] Fetching update data from https://version.home-assistant.io/stable.json,
20-12-03 17:44:06 INFO (MainThread) [supervisor.resolution.fixup] Starting system autofix at state CoreState.RUNNING,
20-12-03 17:44:06 INFO (MainThread) [supervisor.resolution.fixup] System autofix complete,
20-12-03 17:44:07 INFO (MainThread) [supervisor.host.services] Updating service information,
20-12-03 17:44:07 INFO (MainThread) [supervisor.host.network] Updating local network information,
20-12-03 17:44:08 INFO (MainThread) [supervisor.api.security] /dns/info access from a0d7b954_wireguard,
20-12-03 17:44:12 ERROR (MainThread) [asyncio] Task exception was never retrieved,
future: exception=KeyError('ConnectivityCheckEnabled')>,
Traceback (most recent call last):,
File "/usr/src/supervisor/supervisor/host/__init__.py", line 99, in reload,
await self.network.update(),
File "/usr/src/supervisor/supervisor/host/network.py", line 107, in update,
await self.check_connectivity(),
File "/usr/src/supervisor/supervisor/host/network.py", line 76, in check_connectivity,
if not self.sys_dbus.network.connectivity_enabled:,
File "/usr/src/supervisor/supervisor/dbus/network/__init__.py", line 58, in connectivity_enabled,
return self.properties[DBUS_ATTR_CONNECTION_ENABLED],
KeyError: 'ConnectivityCheckEnabled',
20-12-03 17:55:06 ERROR (MainThread) [asyncio] Task exception was never retrieved,
future: ._wrap_task() done, defined at /usr/src/supervisor/supervisor/misc/scheduler.py:58> exception=KeyError('ConnectivityCheckEnabled')>,
Traceback (most recent call last):,
File "/usr/src/supervisor/supervisor/misc/scheduler.py", line 62, in _wrap_task,
await task.coro_callback(),
File "/usr/src/supervisor/supervisor/misc/tasks.py", line 473, in _check_connectivity,
await self.sys_host.network.check_connectivity(),
File "/usr/src/supervisor/supervisor/host/network.py", line 76, in check_connectivity,
if not self.sys_dbus.network.connectivity_enabled:,
File "/usr/src/supervisor/supervisor/dbus/network/__init__.py", line 58, in connectivity_enabled,
return self.properties[DBUS_ATTR_CONNECTION_ENABLED],
KeyError: 'ConnectivityCheckEnabled',

[rnizametdinov]

https://github.com/home-assistant/supervisor/issues/2330 same?

[georaspi]

Very very similar but slightly different final error..

[spurgelaurels]

2330 same?

2288 similar as well.

[ludeeus]

Duplicate of https://github.com/home-assistant/supervisor/issues/2330

[frostworx]

FYI: Arch Linux renames its docker version and therefore also triggers this bug.
Arch package bug is here: https://bugs.archlinux.org/task/68833

Edit: Haven't looked too much around regarding this, but looks like this closed issue sums it up pretty good _(between the lines)_
https://github.com/home-assistant/supervisor/issues/1815

[frostworx]

Arch package maintainer just fixed the PKGBUILD with docker-1:19.03.14-2 :)

[spurgelaurels]

Arch package maintainer just fixed the PKGBUILD with docker-1:19.03.14-2 :)

That was fast.

My suggestion to the HASS developers is to add a passlist to the version strings, so that we're not completely disabling the health checks altogether just to keep our systems alive.

[frostworx]

Yeah, that was really fast :)
Would be nice, if there would be a simple fix upstream, but of course it is their decision.
I understand that not every platform can be supported, but imho there's no reason to explicitly break "unsupported platforms" by rejecting a simple oneliner fix.

[frostworx]

@spurgelaurels
It's not completely fixed yet _(server version still contains -ce)_
edit: now it is!

[spurgelaurels]

My suggestion to the HASS developers is to add a passlist to the version strings, so that we're not completely disabling the health checks altogether just to keep our systems alive.

It has been made clear to me that the HASS developers are not interested in fixing this on their end.

[frenck]

@spurgelaurels That is correct. We are happy to accept bugfixes, but the above-listed systems are not supported by the Home Assistant project.