Core: REST sensors show as "Unavailable" state after upgrading to Home Assistant 0.117.0

Created on 29 Oct 2020 · 31Comments · Source: home-assistant/core

The problem

Hi Guys, after upgrading to Home Assistant 0.117.0 four of my REST sensors started producing an "Unavailable" state along with respective "_failed with [Errno 111] Connect call failed_" errors in the log. My configuration has not changed at all before or after the upgrade.. Also, I can reach ALL the respective API endpoints with a regular web browser without any issues.

Environment

Home Assistant Core release with the issue: Home Assistant 0.117.0
Last working Home Assistant Core release (if known): 0.116
Operating environment (OS/Container/Supervised/Core): VirtualBox VM running on Windows 10
Integration causing this issue: Template
Link to integration documentation on our website: https://www.home-assistant.io/integrations/template

Problem-relevant `configuration.yaml`

sensor:
  - platform: rest
    resource: http://192.168.1.2:61208/api/3/fs
    name: Horizon C Drive Used
    value_template: '{{ value_json[0].percent | round }}'
    unit_of_measurement: "%"
    verify_ssl: false
    timeout: 30
    scan_interval: 16
  - platform: rest
    resource: http://192.168.1.2:61208/api/3/fs
    name: Horizon E Drive Used
    value_template: '{{ value_json[1].percent | round }}'
    unit_of_measurement: "%"
    verify_ssl: false
    timeout: 30
    scan_interval: 16
  - platform: rest
    name: diskio_0
    resource: http://192.168.1.2:61208/api/3/diskio
    value_template: '{{ ( value_json[0] ) }}'
    json_attributes_path: "$.0"
    json_attributes:
      - time_since_update
      - read_bytes
      - write_bytes
    verify_ssl: false
    timeout: 30
    scan_interval: 16
  - platform: rest
    name: diskio_1
    resource: http://192.168.1.2:61208/api/3/diskio
    value_template: '{{ ( value_json[1] ) }}'
    json_attributes_path: "$.1"
    json_attributes:
      - time_since_update
      - read_bytes
      - write_bytes
    verify_ssl: false
    timeout: 30
    scan_interval: 16

It is important to note that I have SEVERAL other REST sensors (few examples below) that connect to the same server that don't have any issues at all. The only difference I see is in the way value_template is defined when parsing a json response with more than 1 key.

  - platform: rest
    resource: http://192.168.1.2:61208/api/3/cpu
    name: Horizon CPU
    value_template: '{{ value_json.total | round }}'
    unit_of_measurement: "%"
    verify_ssl: false
    timeout: 30
    scan_interval: 16
  - platform: rest
    resource: http://192.168.1.2:61208/api/3/mem
    name: Horizon RAM
    value_template: '{{ value_json.percent | round }}'
    unit_of_measurement: "%"
    verify_ssl: false
    timeout: 30
    scan_interval: 16
  - platform: rest
    resource: http://192.168.1.2:61208/api/3/memswap
    name: Horizon Pagefile
    value_template: '{{ value_json.percent | round }}'
    unit_of_measurement: "%"
    verify_ssl: false
    timeout: 30
    scan_interval: 16

```yaml

## Traceback/Error logs
<!--
  If you come across any trace or error logs, please provide them.
-->
I get the below errors constantly in the log file:

2020-10-29 12:04:26 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.1.2:61208/api/3/fs failed with [Errno 111] Connect call failed ('192.168.1.2', 61208)
2020-10-29 12:04:26 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.1.2:61208/api/3/fs failed with [Errno 111] Connect call failed ('192.168.1.2', 61208)
2020-10-29 12:04:26 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.1.2:61208/api/3/diskio failed with [Errno 111] Connect call failed ('192.168.1.2', 61208)
2020-10-29 12:04:26 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.1.2:61208/api/3/diskio failed with [Errno 111] Connect call failed ('192.168.1.2', 61208)

```txt

Additional information

I looked at Home Assistant 0.117.0 breaking changes for templates.. however, I can't tell what change(s) broke my 4 REST sensors. It seems like something changed in Home Assistant adversely affecting REST sensors that have lines like below:

value_template: '{{ ( value_json[0] ) }}'
value_template: '{{ ( value_json[1] ) }}'

rest

Source

mkanet

👍1

Most helpful comment

I'm still stuck using Home Assistant Version: 0.116.x

I hope this helps...

sensor:
  - platform: rest
    resource: http://192.168.1.2:61208/api/3/fs
    name: Horizon C Drive Used
    value_template: '{{ value_json[0].percent | round }}'
    unit_of_measurement: "%"
    verify_ssl: false
    timeout: 30
    scan_interval: 16

out.txt

sensor:
  - platform: rest
    name: diskio_0
    resource: http://192.168.1.2:61208/api/3/diskio
    value_template: '{{ ( value_json[0] ) }}'
    json_attributes_path: "$.0"
    json_attributes:
      - time_since_update
      - disk_name
      - read_bytes
      - write_bytes
    verify_ssl: false
    timeout: 30
    scan_interval: 16

out2.txt

mkanet on 11 Nov 2020

👍2

All 31 comments

rest documentation
rest source
_{^{(message by IssueLinks)}}

probot-home-assistant[bot] on 29 Oct 2020

cogneato on 29 Oct 2020

@cogneato I would like to revert back to version 0.116.4 in order to use my REST sensors. Do you guys have the information you need from me; or, should I wait?

mkanet on 29 Oct 2020

@mkanet I think you've provided what you can as far as I can tell. You should go ahead and downgrade if you need to.

cogneato on 29 Oct 2020

Thank you.

mkanet on 29 Oct 2020

My guess is it is related to: https://github.com/home-assistant/core/pull/41973

You don't happen to be running proxies do you?

elupus on 30 Oct 2020

Scratch that. Seem you say it works with simpler templates.

elupus on 30 Oct 2020

same problem here. downgrade to 0.116.4 solved the problem

fsghzh on 4 Nov 2020

I haven't heard any news about a fix for this yet. I'm still using version 0.116.4

mkanet on 4 Nov 2020

If this is still a problem, please turn on debug logging so we can get more data about why it cannot connect

bdraco on 4 Nov 2020

@bdraco or @cogneato Could one of you please tell me what specific logger settings I should set? I want to be sure I capture all the debug information for the appropriate component(s) since it's a lot of trouble for me to upgrade to 0.117.0 again, capture the log, then restore version 0.116.4 again. Will the below settings capture everything needed related to this issue?

logger:
  default: info
  logs:
    homeassistant.core: debug

I want to be certain I capture all the information you guys need before restoring to version 0.116.4 again.

@fsghzh Maybe, you wouldn't mind also doing the same in order to confirm you're having the same issue?

mkanet on 4 Nov 2020

Hi, I upgraded to 0.117.4. Still the same Problem.
This is one of the sensors:

platform: rest
name: WP_Temperatur_Aussen
resource: "http://192.168.168.24/usr-cgi/xml.cgi?A|27|27"
value_template: "{{ value_json['PCOWEB']['PCO']['ANALOG']['VARIABLE']['VALUE'] }}"
unit_of_measurement: °C
device_class: temperature
scan_interval: 300

Here is the log entry:

2020-11-05 11:42:03 INFO (MainThread) [homeassistant.components.sensor] Setting up sensor.rest
2020-11-05 11:42:03 INFO (MainThread) [homeassistant.components.sensor] Setting up sensor.rest
2020-11-05 11:42:04 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.168.24/usr-cgi/xml.cgi?A|27|27 failed with peer unexpectedly closed connection
2020-11-05 11:42:04 WARNING (MainThread) [homeassistant.components.sensor] Platform rest not ready yet. Retrying in 60 seconds.
2020-11-05 11:42:04 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.168.24/usr-cgi/xml.cgi?A|29|29 failed with peer unexpectedly closed connection
2020-11-05 11:42:04 WARNING (MainThread) [homeassistant.components.sensor] Platform rest not ready yet. Retrying in 60 seconds.
2020-11-05 11:42:05 INFO (MainThread) [homeassistant.components.sensor] Setting up sensor.rest
2020-11-05 11:42:05 INFO (MainThread) [homeassistant.components.sensor] Setting up sensor.rest
2020-11-05 11:42:05 INFO (MainThread) [homeassistant.components.sensor] Setting up sensor.rest
2020-11-05 11:42:05 INFO (MainThread) [homeassistant.components.sensor] Setting up sensor.rest
2020-11-05 11:42:05 INFO (MainThread) [homeassistant.components.sensor] Setting up sensor.rest
2020-11-05 11:42:05 INFO (MainThread) [homeassistant.components.sensor] Setting up sensor.rest
2020-11-05 11:42:05 INFO (MainThread) [homeassistant.components.sensor] Setting up sensor.rest
2020-11-05 11:42:05 INFO (MainThread) [homeassistant.components.sensor] Setting up sensor.rest
2020-11-05 11:42:05 INFO (MainThread) [homeassistant.components.sensor] Setting up sensor.rest
2020-11-05 11:42:05 INFO (MainThread) [homeassistant.components.sensor] Setting up sensor.rest
2020-11-05 11:42:06 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.168.24/usr-cgi/xml.cgi?A|28|28 failed with peer unexpectedly closed connection
2020-11-05 11:42:06 WARNING (MainThread) [homeassistant.components.sensor] Platform rest not ready yet. Retrying in 60 seconds.
2020-11-05 11:42:06 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.168.24/usr-cgi/xml.cgi?A|31|31 failed with peer unexpectedly closed connection
2020-11-05 11:42:06 WARNING (MainThread) [homeassistant.components.sensor] Platform rest not ready yet. Retrying in 60 seconds.
2020-11-05 11:42:11 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.168.24/usr-cgi/xml.cgi?A|30|30 failed with peer unexpectedly closed connection
2020-11-05 11:42:11 WARNING (MainThread) [homeassistant.components.sensor] Platform rest not ready yet. Retrying in 60 seconds.
2020-11-05 11:42:11 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.168.24/usr-cgi/xml.cgi?A|40|40 failed with peer unexpectedly closed connection
2020-11-05 11:42:11 WARNING (MainThread) [homeassistant.components.sensor] Platform rest not ready yet. Retrying in 60 seconds.
2020-11-05 11:42:11 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.168.24/usr-cgi/xml.cgi?A|41|41 failed with peer unexpectedly closed connection
2020-11-05 11:42:11 WARNING (MainThread) [homeassistant.components.sensor] Platform rest not ready yet. Retrying in 60 seconds.
2020-11-05 11:42:12 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.168.24/usr-cgi/xml.cgi?D|44|44 failed with peer unexpectedly closed connection
2020-11-05 11:42:12 WARNING (MainThread) [homeassistant.components.sensor] Platform rest not ready yet. Retrying in 60 seconds.
2020-11-05 11:42:12 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.168.24/usr-cgi/xml.cgi?D|85|85 failed with peer unexpectedly closed connection
2020-11-05 11:42:12 WARNING (MainThread) [homeassistant.components.sensor] Platform rest not ready yet. Retrying in 60 seconds.
2020-11-05 11:42:12 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.168.24/usr-cgi/xml.cgi?D|80|80 failed with peer unexpectedly closed connection
2020-11-05 11:42:12 WARNING (MainThread) [homeassistant.components.sensor] Platform rest not ready yet. Retrying in 60 seconds.
2020-11-05 11:42:12 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.168.24/usr-cgi/xml.cgi?D|82|82 failed with peer unexpectedly closed connection
2020-11-05 11:42:12 WARNING (MainThread) [homeassistant.components.sensor] Platform rest not ready yet. Retrying in 60 seconds.
2020-11-05 11:42:12 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.168.24/usr-cgi/xml.cgi?D|84|84 failed with peer unexpectedly closed connection
2020-11-05 11:42:12 WARNING (MainThread) [homeassistant.components.sensor] Platform rest not ready yet. Retrying in 60 seconds.
2020-11-05 11:42:12 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.168.24/usr-cgi/xml.cgi?D|136|136 failed with peer unexpectedly closed connection
2020-11-05 11:42:12 WARNING (MainThread) [homeassistant.components.sensor] Platform rest not ready yet. Retrying in 60 seconds.
2020-11-05 11:42:12 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.168.24/usr-cgi/xml.cgi?D|140|140 failed with peer unexpectedly closed connection
2020-11-05 11:42:12 WARNING (MainThread) [homeassistant.components.sensor] Platform rest not ready yet. Retrying in 60 seconds.

fsghzh on 5 Nov 2020

Can you check to see if the webserver at 192.168.168.24 is using proper line endings? It might be this case https://github.com/python-hyper/h11/issues/7

bdraco on 5 Nov 2020

It may be related to this issue open here https://github.com/encode/httpx/issues/1378

bdraco on 5 Nov 2020

@bdraco if you still need me to provide a debug log, can you please let me know which logger settings to use to focus on the components involved? I can upgrade to test if necessary.

mkanet on 5 Nov 2020

Can you check to see if the webserver at 192.168.168.24 is using proper line endings? It might be this case python-hyper/h11#7

Hi,
I did :
curl 'http://192.168.168.24/usr-cgi/xml.cgi?A|27|27' >out.txt

So you should see all the bytes in the response.

out.txt

fsghzh on 6 Nov 2020

@fsghzh you need to get headers too so something like:
curl -ivs --raw http://192.168.168.24/usr-cgi/xml.cgi?A%7C27%7C27 > out.txt

elupus on 6 Nov 2020

@fsghzh you need to get headers too so something like:
curl -ivs --raw http://192.168.168.24/usr-cgi/xml.cgi?A%7C27%7C27 > out.txt

Thanks for the help :

here is the new output of the curl:

out.txt

fsghzh on 6 Nov 2020

I have the same issue with rest sensors with one of the end points within my setup. Hopefully the following information helps you to find the issue.
HA version 116.4 and before is working fine, since 117.0 to 117.4 some rest sensors are not working anymore. After rollback to 116.4 all sensors work again.

End point BSB LAN (handbook)
not working!
configuration.yaml:

sensor:
  - platform: rest
    resource: http://192.168.10.8/xxxxx/JQ=8300
    name: BSB Brennerstufe 1
    value_template: '{{ value_json["8300"]["desc"] }}'

Logs:

2020-11-06 15:03:36 ERROR (MainThread) [homeassistant.components.rest.data] Error fetching data: http://192.168.10.8/xxxxx/JQ=8300 failed with peer unexpectedly closed connection
2020-11-06 15:03:36 WARNING (MainThread) [homeassistant.components.sensor] Platform rest not ready yet. Retrying in 60 seconds.

out.txt

Homematic CCU2 and Reolink Cameras
The rest sensors are working fine, also under HA version 117.x

enjoysimpson on 6 Nov 2020

I'm still stuck using Home Assistant Version: 0.116.x

I hope this helps...

sensor:
  - platform: rest
    resource: http://192.168.1.2:61208/api/3/fs
    name: Horizon C Drive Used
    value_template: '{{ value_json[0].percent | round }}'
    unit_of_measurement: "%"
    verify_ssl: false
    timeout: 30
    scan_interval: 16

out.txt

sensor:
  - platform: rest
    name: diskio_0
    resource: http://192.168.1.2:61208/api/3/diskio
    value_template: '{{ ( value_json[0] ) }}'
    json_attributes_path: "$.0"
    json_attributes:
      - time_since_update
      - disk_name
      - read_bytes
      - write_bytes
    verify_ssl: false
    timeout: 30
    scan_interval: 16

out2.txt

mkanet on 11 Nov 2020

👍2

I'll "me too" this; although I have complex value templates, these were working on the previous versions and stopped on upgrade to 0.116.x+

richard-parker on 11 Nov 2020

I faced this problem with HA 0.117 but it was due to the poor format used by my web server providing the HTTP reply (a socat hackish command). It was the lack of the standard "\r\n" at the end of headers line. It is a known "bug" of python-hyper (https://github.com/python-hyper/h11/issues/7) but a PR is coming (https://github.com/python-hyper/h11/pull/115). For me it was quicker to fix the web server.

mario-tux on 11 Nov 2020

@mario-tux thanks. Unfortunately, this isnt something I can fix in my case. I have no control over Glances's API responses.

mkanet on 11 Nov 2020

@mkanet I'm using Glances too but using internal integration: no problem in getting sensors without rest sensors.

mario-tux on 11 Nov 2020

I can't use the Glances integration in Home Assistant since it only monitors a limited number of resources. It doesn't monitor items like network, disk, gpu, etc

mkanet on 11 Nov 2020

I also have a problem with some rest sensors. After upgrading 0.116.4 to 0.117.6.
first 4 sensors are ok. last 3 sensors not ok So i had to roll-back to 0.116.4.

platform: rest
name: consumption_kwh_low
resource: http://192.168.1.76/api/v1/smartmeter
value_template: '{{ value_json[0][3] }}'
unit_of_measurement: "kWh"
scan_interval: 10
platform: rest
name: consumption_kwh_high
resource: http://192.168.1.76/api/v1/smartmeter
value_template: '{{ value_json[0][4] }}'
unit_of_measurement: "kWh"
scan_interval: 10
platform: rest
name: production_kwh_low
resource: http://192.168.1.76/api/v1/smartmeter
value_template: '{{ value_json[0][5] }}'
unit_of_measurement: "kWh"
scan_interval: 10
platform: rest
name: production_kwh_high
resource: http://192.168.1.76/api/v1/smartmeter
value_template: '{{ value_json[0][6] }}'
unit_of_measurement: "kWh"
scan_interval: 10
platform: rest
name: consumption_w
resource: http://192.168.1.76/api/v1/smartmeter
value_template: '{{ value_json[0][8] }}'
unit_of_measurement: "W"
scan_interval: 10
platform: rest
name: production_w
resource: http://192.168.1.76/api/v1/smartmeter
value_template: '{{ value_json[0][9] }}'
unit_of_measurement: "W"
scan_interval: 10
platform: rest
name: consumption_gas_m3
resource: http://192.168.1.76/api/v1/smartmeter
value_template: '{{ value_json[0][10] }}'
unit_of_measurement: "m3"
scan_interval: 10

GreJ0 on 15 Nov 2020

Does anybody know how this will continue? A change/fix seems not to be part of Home Assistant 0.118.0. I tried it out, same REST sensors are dead after update to 118.0, same behavior as in 117.x. So I am back to 116.4.

enjoysimpson on 19 Nov 2020

@enjoysimpson I'm not sure either. I haven't been able to get a definitive answer.

mkanet on 19 Nov 2020

👍1

We are waiting for https://github.com/python-hyper/h11/pull/115 which appears is actively being reviewed now (within the last 24 hours)

bdraco on 19 Nov 2020

Thanks for the update @bdraco!

mkanet on 19 Nov 2020

@bdraco It appears that the PR was approved. My apologies for my ignorance, but do you know which version of Home Assistant we could see the fix for this issue?

mkanet on 27 Nov 2020

Was this page helpful?

0 / 5 - 0 ratings