Core: Certificate Expiry component

[vlad36N]

Home Assistant release with the issue:

Version 0.99.0

Last working Home Assistant release (if known):
n/a

Operating environment (Hass.io/Docker/Windows/etc.):

arch | x86_64
dev | false
docker | true
hassio | true
os_name | Linux
python_version | 3.7.4
timezone | America/New_York
version | 0.99.0
virtualenv | false

Component/platform:

https://www.home-assistant.io/components/cert_expiry/

Description of problem:
Configuring the sensor using YAML certainly not working.
Manually add it + bottom right. Certificate.
But every time HA restart sensor shows “Unavailable” status (adding new sensor Obihai ).
Have to remove from integration, restart, add again.

Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):

sensor:
  - platform: cert_expiry
    host: xxxxxx.xxxxxx.xxxxx

Traceback (if applicable):

Additional information:

[probot-home-assistant[bot]]

Hey there @cereal2nd, mind taking a look at this issue as its been labeled with a integration (cert_expiry) you are listed as a codeowner for? Thanks!

[tomlut]

Last working version was 0.98.5

Discovery is ignoring my sensor defined in my configuration.yaml in 0.99.2 and I have to use the discovered integration instead. This did not occur in 0.98.5

sensor:
  - platform: cert_expiry
    host: mydomain.duckdns.org

[hellcry37]

Same problem here:
When I set it up it works until restart, after that is always unavailable as a sensor.
Also setting up integration via config files doesn't work anymore and pops a notification:
"We have discovered new devices on your network. Check it out"

For now I removed all my config entry's and setup integration directly in integrations page that works until restart, after restart this happens:

2019-09-21 08:19:58 ERROR (SyncWorker_38) [homeassistant.components.cert_expiry.sensor] Cannot fetch certificate from xxxxxxxxx
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/cert_expiry/sensor.py", line 102, in update
    cert = get_cert(self.server_name, self.server_port)
  File "/usr/src/homeassistant/homeassistant/components/cert_expiry/helper.py", line 12, in get_cert
    with socket.create_connection(address, timeout=TIMEOUT) as sock:
  File "/usr/local/lib/python3.7/socket.py", line 727, in create_connection
    raise err
  File "/usr/local/lib/python3.7/socket.py", line 716, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
2019-09-21 08:20:16 ERROR (SyncWorker_16) [homeassistant.components.cert_expiry.sensor] Cannot fetch certificate from xxxxxxxxx
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/cert_expiry/sensor.py", line 102, in update
    cert = get_cert(self.server_name, self.server_port)
  File "/usr/src/homeassistant/homeassistant/components/cert_expiry/helper.py", line 12, in get_cert
    with socket.create_connection(address, timeout=TIMEOUT) as sock:
  File "/usr/local/lib/python3.7/socket.py", line 727, in create_connection
    raise err
  File "/usr/local/lib/python3.7/socket.py", line 716, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

[hellcry37]

P.S. I use a custom port forwarded in router (that used to work OK until 0.99 release) not the default one.

[tomlut]

When I set it up it works until restart, after that is always unavailable as a sensor.

It only updates every 12 hours or so to prevent spamming the server. It will eventually resolve from 'Unavailable' to a value.

EDIT: actually that may not be true. Unless it is limited on the server side because like you I too am seeing the connection refused error on restarting.

I think it must be server side limited as it does become available after 12 hours.

[hellcry37]

@tomlut i can assure you this is not the problem. Do this:

• remove integration
• do a restart of ha to make sure all ok
• add integration from integration page

You will see it will work right away until you restart again. You can do this every 5 mins if you want it will work every time.

Even if it were a server side problem, sensor should keep old value in my logic for at least 12 hours until is sure new server value was pulled.

[Madelinot]

Something was changed for sure because my sensor stopped working as well. I had the certificate expiry sensor defined in my yaml configuration and it's been happy for a long time. With 99.x, it stopped working and I got the Integration discovered message. I proceeded to do it and it worked right away, until I restarted; then it would not work anymore. I removed EVERYTHING (integration and yaml config) and just let it HA discover it. I now get "unavailable" for the value of the certificate. Something is wrong for sure...

[Madelinot]

... FYI, I do get the "[Errno 111] Connection refused" message in my logs.

[jjlawren]

I have quite a few certs that I'm monitoring and I haven't noticed any issues in the upgrade to 0.99.X along with quite a few restarts after the upgrade. This doesn't seem like a general problem that affects every setup.

1) The component will import a given YAML config exactly once, but after that point it will load from the saved config entry (<CONFIG>/.storage/core.config_entries). These will show up on the Integrations page. A YAML config will _not_ be used if its host:port combination is being used by an existing config entry. Deleting the config entry will cause it to load from YAML on the next restart.
2) Is there a pattern in the hostnames everyone is using? I assume nobody is using raw IPs as we're talking about validating certs? Could there be some DNS weirdness going?
3) Is this happening to anyone not running Hass.io?

[Madelinot]

I have two instances of HASS.io in two different locations. They are almost identical in terms of setup. One is experiencing the issue described, the other seems fine. They both have valid cert with a registered name.

[Cereal2nd]

i added some more exceptions and more error logging to the component in the above commit
can someone who has the problem test that one out?

it should print more error cases in the logs

[hellcry37]

i'll help you test it if you tell me what you need me to do

[Cereal2nd]

apply the following patch to:
homeassistan/components/cert_expiry/sensor.py

https://github.com/Cereal2nd/home-assistant/commit/7ce7e5f2fdd391710b5a3a6104b9d2ac8a8a1a5f

[hellcry37]

Can't find it tbh, I use hass.io so what you want me to replace is in docker container for sure. can i do a custom component ?

[Cereal2nd]

in hass.io i have no clue where the files are :(

[hellcry37]

nvm i edit it in docker, i'll restart now

p.s. the path is:
/usr/src/homeassistant/homeassistant/components/cert_expiry/sensor.py

[hellcry37]

error is now:
2019-10-02 21:51:12 ERROR (SyncWorker_4) [homeassistant.components.cert_expiry.sensor] Got a connection refused error from mydomanin.ro:58000

[hellcry37]

i used this in sensor.py
https://raw.githubusercontent.com/Cereal2nd/home-assistant/7ce7e5f2fdd391710b5a3a6104b9d2ac8a8a1a5f/homeassistant/components/cert_expiry/sensor.py

[jjlawren]

@hellcry37 is that a valid DNS name? Does not resolve with public DNS servers.

[hellcry37]

You mean mydomanin.ro:58000 ? NO, is just an example, I cant put my real domain name and port here for security reasons

[hellcry37]

but i can assure you my domain name is valid and works as intended, that's not an issue

[jjlawren]

That message is only printed when a ConnectionRefusedError exception is thrown. This seems like a networking-related issue, not with the component itself. Could be related to the DNS changes in Hass.io from a few releases back, could be something else.

[hellcry37]

okayyyyyyy, explain this pls:

• I've put back the original sensor.py in docker
• Removed certificate integration
• Restarted
• add it back after restart and it works ok (shows 7 days)
• now if i restart it will show again “Unavailable”

[hellcry37]

this is at least weird, how else i can help?

[jjlawren]

That is weird. Can you compare your YAML config to the contents of the .storage/core.config_entries file to see if there's an obvious difference that could cause a problem?

[hellcry37]

yaml conf is commented because it doesn't work anymore, if i use it without setting up component in integrations, then i only see a notification i should setup cert in integrations
looks like this:

# Lets Encript Certificate Expire Sensor
  # - platform: cert_expiry
  #   host: !secret url_ha_cert
  #   port: !secret port_ha_cert

Where secret file looks like this:

# SSL Certificate
# url_ha_cert: subdomanin.go.ro
# port_ha_cert: 58000

Anyway like i said all commented because it does not work from yaml like it worked before

and core.config_entries looks like this:

            {
                "connection_class": "local_poll",
                "data": {
                    "host": "subdomanin.go.ro",
                    "port": 58000
                },
                "domain": "cert_expiry",
                "entry_id": "id_here",
                "options": {},
                "source": "user",
                "system_options": {
                    "disable_new_entities": false
                },
                "title": "ssl_certificate_expiry",
                "version": 1
            }

[Madelinot]

okayyyyyyy, explain this pls:

• I've put back the original sensor.py in docker
• Removed certificate integration
• Restarted
• add it back after restart and it works ok (shows 7 days)
• now if i restart it will show again “Unavailable”

I get the exact same behavior on mine...

[MartinHjelmare]

We do an update on home assistant start:
https://github.com/home-assistant/home-assistant/blob/9c1feacd47671eb4935215c31110b778f3a63eb5/homeassistant/components/cert_expiry/sensor.py#L93-L97

We also do an update when the entity is added:
https://github.com/home-assistant/home-assistant/blob/9c1feacd47671eb4935215c31110b778f3a63eb5/homeassistant/components/cert_expiry/sensor.py#L47-L50

So on restart two updates will happen with close proximity in time. Maybe that's not good?

[tomlut]

So there are two issues.

1. The yaml configuration is no longer recognised.
2. The discovered certificate sensor becomes unavailable after a restart. On my system at least, the discovered sensor is updated 12 hours (thereabouts, I've never timed it) after a restart.

[jjlawren]

I _believe_ the linked PR above should take care of this, even though I could not reproduce the problem. If anyone affected is comfortable testing before this is merged, I'd appreciate it.

[hellcry37]

i can try to test it if you tell me what to change and where

[jjlawren]

@hellcry37 This is the fixed version of the sensor.py file that will be in the next release: https://raw.githubusercontent.com/home-assistant/home-assistant/bb45bdd8dd937a9fbbb21def1da93299498a9290/homeassistant/components/cert_expiry/sensor.py

[hellcry37]

yep just test it seem to work just fine, thanks

[hellcry37]

BTW: Thanks for your work.

[Madelinot]

I don't see this fix committed yet for the next release (0.100), as per: https://rc.home-assistant.io/blog/2019/10/03/release-100/

In which version will it be included?

Thanks!

[jjlawren]

It will be in 0.100.

[ttaidapos]

Issues starting with 103.1-5
Below is what I get when i make the declaration in sensor.yaml and then try to add it through integrations, don't get the popup to enter URL/Port or anything, it just hangs. You even get a "discovered new devices" with the cert_expiry once you add it to sensor.yaml. See below error.

What I tried:
Removing from sensor.yaml, adding through integrations (if you don't have it defined in sensor.yaml it works fine), stopping services, adding back to sensor.yaml, starting services.

Result
Nothing, it boots up with "unavailable". Removing/adding the integration fixes the issue but the problem is other integration depend on this during bootup. For instance, crime_reports depends on this to even pull data.

Having just the integration and no sensor.yaml, after restarting services you get "unavailable" again.

Dec 30 23:09:53 raspberrypi hass[21472]: 2019-12-30 23:09:53 ERROR (MainThread) [aiohttp.server] Error handling request
Dec 30 23:09:53 raspberrypi hass[21472]: Traceback (most recent call last):
Dec 30 23:09:53 raspberrypi hass[21472]: File "/srv/homeassistant/lib/python3.6/site-packages/aiohttp/web_protocol.py", line 418, in start
Dec 30 23:09:53 raspberrypi hass[21472]: resp = await task
Dec 30 23:09:53 raspberrypi hass[21472]: File "/srv/homeassistant/lib/python3.6/site-packages/aiohttp/web_app.py", line 458, in _han dle
Dec 30 23:09:53 raspberrypi hass[21472]: resp = await handler(request)
Dec 30 23:09:53 raspberrypi hass[21472]: File "/srv/homeassistant/lib/python3.6/site-packages/aiohttp/web_middlewares.py", line 119, in impl
Dec 30 23:09:53 raspberrypi hass[21472]: return await handler(request)
Dec 30 23:09:53 raspberrypi hass[21472]: File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/components/http/real_ip. py", line 40, in real_ip_middleware
Dec 30 23:09:53 raspberrypi hass[21472]: return await handler(request)
Dec 30 23:09:53 raspberrypi hass[21472]: File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/components/http/ban.py", line 73, in ban_middleware
Dec 30 23:09:53 raspberrypi hass[21472]: return await handler(request)
Dec 30 23:09:53 raspberrypi hass[21472]: File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/components/http/auth.py" , line 136, in auth_middleware
Dec 30 23:09:53 raspberrypi hass[21472]: return await handler(request)
Dec 30 23:09:53 raspberrypi hass[21472]: File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/components/http/view.py" , line 123, in handle
Dec 30 23:09:53 raspberrypi hass[21472]: result = await result
Dec 30 23:09:53 raspberrypi hass[21472]: File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/components/config/config _entries.py", line 154, in get
Dec 30 23:09:53 raspberrypi hass[21472]: return await super().get(request, flow_id)
Dec 30 23:09:53 raspberrypi hass[21472]: File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/helpers/data_entry_flow. py", line 78, in get
Dec 30 23:09:53 raspberrypi hass[21472]: result = await self._flow_mgr.async_configure(flow_id)
Dec 30 23:09:53 raspberrypi hass[21472]: File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/data_entry_flow.py", lin e 86, in async_configure
Dec 30 23:09:53 raspberrypi hass[21472]: if cur_step.get("data_schema") is not None and user_input is not None:
Dec 30 23:09:53 raspberrypi hass[21472]: AttributeError: 'NoneType' object has no attribute 'get'

[jjlawren]

@ttaidapos this is a closed issue. Please open a new issue if you're experiencing new problems.