Core: HomeMatic climate, cover and light devices not working in 0.104.0b3

[xiidoz]

Home Assistant release with the issue:
0.104.0b3

Last working Home Assistant release (if known):
0.103.6

Operating environment (Hass.io/Docker/Windows/etc.):
Docker

Integration:
HomeMatic (https://rc.home-assistant.io/integrations/homematic/)

Description of problem:
I'm using a RasperryMatic on latest firmware (3.49.17.20191225) with HmIP-RFUSB (HomeMatic IP devices only). With the update to 0.104.0b3 all homematic climate devices show up as unavailable.
I tried all three possible port configuration options in the hosts section:

• no port configuration at all (default 2001 should have been applied)
• port: 2001
• port: 2010

None of those three worked.

Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):

homematic:
  local_ip: 10.10.1.3
  interfaces:
    ip:
      host: homematic.devices
      port: 2010
      callback_ip: 10.10.1.3
      username: !secret hm_username
      password: !secret hm_password
    groups:
      host: homematic.devices
      port: 9292
      callback_ip: 10.10.1.3
      username: !secret hm_username
      password: !secret hm_password
      path: /groups
  hosts:
    ccu3:
      host: homematic.devices
      port: 2010
      username: !secret hm_username
      password: !secret hm_password

Traceback (if applicable):

Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 344, in _async_add_entity
    capabilities=entity.capability_attributes,
  File "/usr/src/homeassistant/homeassistant/components/climate/__init__.py", line 179, in capability_attributes
    ATTR_HVAC_MODES: self.hvac_modes,
  File "/usr/src/homeassistant/homeassistant/components/homematic/climate.py", line 87, in hvac_modes
    if "AUTO_MODE" in self._hmdevice.ACTIONNODE:
AttributeError: 'NoneType' object has no attribute 'ACTIONNODE'

Additional information:
All my other devices (sensors, binary_sensors) are functioning properly.

[probot-home-assistant[bot]]

Hey there @pvizeli, @danielperna84, mind taking a look at this issue as its been labeled with a integration (homematic) you are listed as a codeowner for? Thanks!

[danielperna84]

Which types of Climate devices are you using? With pyhomematic version 0.1.62 there have only been changes to HmIP-BWTH and HmIP-BWTH24. With 0.1.63 all the eTRV-Variants had a small change. But there actually only a new a new attribute in the writenode has been added, so this shouldn't have any effect.

[xiidoz]

I'm having multiple HmIP-eTRV-2, HmIP-WTH-2 and heating control groups containing the former. The error message from above is thrown for each of those devices/groups.
It's working fine with Home Assistant release 0.103.6, so pyhomematic 0.1.62 doesn't seem to be the problem. I've also tried HA 0.104.0b4 with earlier pyhomematic versions, which all threw the same error 'NoneType' object has no attribute 'ACTIONNODE'. So my guess is, it's somewhere in HA and not in pyhomematic.

[danielperna84]

It's also 0.1.62 in 0.104.0b4. The strange thing is, that none of the recent changes have to do anything with what's happening here.
The next best thing that comes to my mind are the changes in the climate component itself. They came with PR #30544.

[danielperna84]

I quickly have set up a new dev environment and can confirm the issue. And it's not just HmIP. I only have regular HM-thermostats, and here I have the same problem.

[danielperna84]

The entities where loaded after implementing a check for self._hmdevice HomeMatics climate.py:

    @property
    def hvac_modes(self):
        """Return the list of available hvac operation modes.

        Need to be a subset of HVAC_MODES.
        """
        if self._hmdevice is None:
            return [HVAC_MODE_HEAT, HVAC_MODE_OFF]
        if "AUTO_MODE" in self._hmdevice.ACTIONNODE:
            return [HVAC_MODE_AUTO, HVAC_MODE_HEAT, HVAC_MODE_OFF]
        return [HVAC_MODE_HEAT, HVAC_MODE_OFF]
...
...
    @property
    def preset_modes(self):
        """Return a list of available preset modes."""
        preset_modes = []
        if self._hmdevice is None:
            return preset_modes
        for mode in self._hmdevice.ACTIONNODE:
            if mode in HM_PRESET_MAP:
                preset_modes.append(HM_PRESET_MAP[mode])
        return preset_modes

But this obviously is not a fix, it just hides the error and let's the startup process continue.

While doing this I also noticed that the cover platform doesn't work anymore. I get the following error:

2020-01-13 22:24:23 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/home/daniel/home-assistant/homeassistant/helpers/entity_platform.py", line 345, in _async_add_entity
    supported_features=entity.supported_features,
  File "/home/daniel/home-assistant/homeassistant/components/cover/__init__.py", line 207, in supported_features
    if self.current_cover_position is not None:
  File "/home/daniel/home-assistant/homeassistant/components/homematic/cover.py", line 39, in current_cover_position
    return int(self._hm_get_state() * 100)
TypeError: unsupported operand type(s) for *: 'NoneType' and 'int'

And by testing with pydevccu I also found the light platform to be broken:

2020-01-13 22:41:24 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/home/daniel/home-assistant/homeassistant/helpers/entity_platform.py", line 344, in _async_add_entity
    capabilities=entity.capability_attributes,
  File "/home/daniel/home-assistant/homeassistant/components/light/__init__.py", line 391, in capability_attributes
    supported_features = self.supported_features
  File "/home/daniel/home-assistant/homeassistant/components/homematic/light.py", line 58, in supported_features
    if "COLOR" in self._hmdevice.WRITENODE:
AttributeError: 'NoneType' object has no attribute 'WRITENODE'

@pvizeli Are you aware of any core changes that might have had such a great impact?

[danielperna84]

Instructions on how to reproduce (without owning HomeMatic devices):

Home Assistant configuration

homematic:
  interfaces:
    hmrfdev:
      host: 127.0.0.1

Install virtual CCU

pip3 install pydevccu

Start (and stop) virtual CCU

import pydevccu
# Emulate Thermostat, Cover and Light
s = pydevccu.Server(devices=["HM-CC-RT-DN", "HM-LC-Bl1-SM", "HM-LC-RGBW-WM"])
s.start()
# Start Home Assistant with HM interface at 127.0.0.1:2001
s.stop() # When done testing

[pvizeli]

There is an core issue and flow change. Will fix that tomorrow. Thanks for debug it

[balloob]

The problem what happened is that we've started storing capability_attributes in the entity registry.

The capability_attributes are fetched from an entity after the initial update is done (if requested) but before we execute async_added_to_hass.