Core: Update for cover.garageopener fails - TypeError: can't concat bool to bytearray

[evisser4]

Home Assistant release with the issue:
0.88.2

Last working Home Assistant release (if known):

Operating environment (Hass.io/Docker/Windows/etc.):
Linux subsystem for windows 10

Component/platform:
https://www.home-assistant.io/components/homekit_controller/

Description of problem:
About 1 hour after Home Assistant startup I start getting errors every minute or so, like the one reported below, keep appearing in home-assistant log. Sometimes my homekit enabled garage door keeps functioning, sometimes it looks like the home-assistant UI Cover state gets out of sync with the actual state of the garage door (and therefore open/close commands don't work anymore). Usually, restart of home-assistant "fixes" the errors from log, but after about an hour they start appearing again.

Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):

# Discover some devices automatically
discovery:
  enable:
    - homekit

Traceback (if applicable):

Mon Mar 04 2019 11:40:42 GMT-0500 (Eastern Standard Time)
Update for cover.garageopener fails
Traceback (most recent call last):
  File "/home/srednas/.local/lib/python3.6/site-packages/homeassistant/helpers/entity.py", line 221, in async_update_ha_state
    await self.async_device_update()
  File "/home/srednas/.local/lib/python3.6/site-packages/homeassistant/helpers/entity.py", line 349, in async_device_update
    await self.hass.async_add_executor_job(self.update)
  File "/usr/lib/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/srednas/.local/lib/python3.6/site-packages/homeassistant/components/homekit_controller/__init__.py", line 267, in update
    new_values_dict = pairing.get_characteristics(self._chars_to_poll)
  File "/home/srednas/.local/lib/python3.6/site-packages/homekit/controller.py", line 375, in get_characteristics
    response = self.session.get(url)
  File "/home/srednas/.local/lib/python3.6/site-packages/homekit/controller.py", line 637, in get
    return self.sec_http.get(url)
  File "/home/srednas/.local/lib/python3.6/site-packages/homekit/http_impl/secure_http.py", line 66, in get
    return self._handle_request(data)
  File "/home/srednas/.local/lib/python3.6/site-packages/homekit/http_impl/secure_http.py", line 92, in _handle_request
    return self._read_response()
  File "/home/srednas/.local/lib/python3.6/site-packages/homekit/http_impl/secure_http.py", line 159, in _read_response
    response.parse(decrypted)
  File "/home/srednas/.local/lib/python3.6/site-packages/homekit/http_impl/response.py", line 38, in parse
    self._raw_response += part
TypeError: can't concat bool to bytearray

Additional information:

[evisser4]

@Jc2k

[Jc2k]

Hi @evisser4, thanks for the ping.

Are there any other exceptions? This exception usually appears as a seconday exception after a connection error of some kind.

What normally happens is the TCP connection drops after as little as 15m, but some times a couple of hours. This manifests in python as an OSError or a socket.error. On the latest HA/homekit_python those errors get translated into an AccessoryDisconnectedError, this is handled cleanly and the session will reconnect.

The error you are seeing normally happens when that initial failure is not handled properly and things continue to used the failed TCP session. This is probably something to do with WSL, which is not something we have ever tested with AFAIK. There is handling for this in the upstream libray now to stop the TypeError, but it will be a little while before it makes it into a release.

Another possible cause is on bridges with more than 1 accessory connected, although thats only theoretical so far - I have not seen it generate this error in this case before.

[evisser4]

There are no other exceptions that I can see. I am only looking at the home assistant log. Is there another place I can look for exceptions? To be clear, you are suggesting maybe I would not see this issue if I ran home assistant on a different os than linux subsystem? I could try installing it on a different system.

Thanks for your input.

[Jc2k]

It would normally be that there was a bunch of TypeError's in the home assistant log, and then right before the first TypeError there would be a OSError or similar.

I can't guarantee that WSL is the problem, but it is not something I have seen successfully used before - it certainly seems like something to rule out if possible.

[evisser4]

Yeah, no TypeErrors or OSError in my home assistant log. I will try to setup on a different system to rule that out.

[evisser4]

@Jc2k I set up a new home assistant using the vdk in virtual box on my windows machine. I am getting the same error on there as well. Any ideas?

[Jc2k]

We are off the beaten path i'm afraid. Some ideas:

• Maybe it's the same problem we are used to but for some reason in your network or with your device or something else we haven't seen before we are skipping the OSError i'm expecting and going to the straight to the TypeError. This would be good in that the next release of homekit_python will fix it. But bad in that I don't know how that could happen.

• I don't know what kind of device you have but if there were multiple entities for it (either its a bridge with multiple accessories or a single accessory that exports multiple services/entities) then i'd expect weird errors like this. There is actually insufficient locking in this component which means concurrent attempts to use the secure session can cause crypto errors. I have a branch that adds the locking but it's going to take a while to clear review and be merged.

• Are there any other steps or processes in your environment that I might need to fully recreate this problem? E.g. are you using homekit_controller and homekit? (I.e. do you export your HA entities to an iPhone/iPad/Siri etc). I guess the same question applies to any component that exposes your HA to other ecosystems - like the emulated hue component or Alexa integration or any thing that uses HA API's? What about automations?

[evisser4]

Well, I am using homekit_controller AND homekit. But to be sure it was unrelated, I just did a clean install and only paired my garage opener with homekit_controller. This did not seem to make a difference. So it is safe to assume the errors are not homekit related.

This is a single accessory, An insignia garage door opener.

I updated homekit_python to the latest master and I am giving that a run now...I noticed it includes better handling of the decryption failures. I will report back.

[Jc2k]

Yeah that’s the fix I mean. It won’t stop an error appearing in your logs (on its own) but it should allow HA to recover without needing a restart.

[evisser4]

To answer your earlier question, To reproduce the issue, I do a clean install, edit configuration.yaml for discovery homekit. Then pair the Insignia Garage Door Opener. Then I don't even need to open or close the garage door. Just sit and wait for 30 or so minutes idle and it will throw the error.

[evisser4]

I can confirm that home assistant no longer freezes after updating homekit_python. As you said the errors are still present. Do you know if these updates for error handling are included in 0.89?

[Jc2k]

No - they require a new release of homekit_python and then some more work on the HA side (to fully handle the new EncryptionError). Not sure when that will be - upstream is very busy - but it is in the pipeline.

[Jc2k]

Upstream has released 0.12.3 which has a part of the fix in. #21793 references a PR to update HA to use it.

[Jc2k]

I believe the fix for this landed in dev with #21965. I think we can close this - if there are other issues feel free to tag me in a new issue!