Core: Somfy API integration bug: TypeError: unsupported operand type(s) for -: 'int' and 'str'

[pascalsaul]

Home Assistant release with the issue:
0.103.2

Last working Home Assistant release (if known):
0.102.x (9 dec)

Maybe since this commit was in the release: https://github.com/home-assistant/home-assistant/pull/29675 or maybe Somfy changed something on their end but can't find any release notes.

Operating environment (Hass.io/Docker/Windows/etc.):
Docker

Integration:
https://www.home-assistant.io/integrations/somfy/

Description of problem:

Shutter component not working anymore (roller_shutter_positionable_stateful_generic)
Screen does work (exterior_blind_positionable_stateful_generic)

Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):

somfy:
  client_id: xxx
  client_secret: xxx

Traceback (if applicable):

2019-12-19 12:45:38 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 286, in async_update_ha_state
    self._async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 327, in _async_write_ha_state
    attr.update(self.state_attributes or {})
  File "/usr/src/homeassistant/homeassistant/components/cover/__init__.py", line 198, in state_attributes
    current = self.current_cover_position
  File "/usr/src/homeassistant/homeassistant/components/somfy/cover.py", line 69, in current_cover_position
    position = 100 - self.cover.get_position()
TypeError: unsupported operand type(s) for -: 'int' and 'str'

Additional information:

[rabesocke]

I can confirm this issue. Probably this has nothing to do with HA because I had 0.103.0 running without problems. The problem occurred today for the first time.

[Dulcow]

The problem appeared on my Raspberry Pi 3 days ago. Today, my second instance of Docker got the same treatment. I don't know if there is something else we can do to help troubleshoot this one.

[probot-home-assistant[bot]]

Hey there @tetienne, mind taking a look at this issue as its been labeled with a integration (somfy) you are listed as a codeowner for? Thanks!

[springstan]

self.cover.get_position() seems to return a string instead of an int, although the used library explicitly casts the value to an integer, see: https://github.com/tetienne/somfy-open-api/blob/c4b9f32203380f49db6303ca77231f543d303c7b/pymfy/api/devices/roller_shutter.py#L10-L11

This problem could be solved by adding another cast, e.g. int(self.cover.get_position()).
However, this would add another cast which is bad practice since it is unnecessary.
@tetienne hope this helps you in the debugging process ✌

[rabesocke]

I found the same possible solution and tested it in my Docker installation.

I changed in homeassistant/components/somfy/cover.py the line 69 to
position = 100 - int(self.cover.get_position())

The problem in the library could be that the cast()-method of the typing library only cast the value but do not convert it as int(). Maybe int() should be used to convert the value to an Integer.
But I do not have too much experience with Python in that case.

typing.cast(typ, val)
Cast a value to a type.
This returns the value unchanged. To the type checker this signals that the return value has the designated type, but at runtime we intentionally don’t check anything (we want this to be as fast as possible).
https://docs.python.org/3/library/typing.html#typing.cast

[springstan]

@rabesocke did adding another cast work?

[rabesocke]

Yes, the change that I posted works at the moment in my installation. Sorry, I wrote it a little indistinctly.

[rabesocke]

Another hint for finding the real problem: The error only occures when the cover is completely open - any other position don't raise the error in my installation.

[pascalsaul]

Another hint for finding the real problem: The error only occures when the cover is completely open - any other position don't raise the error in my installation.

If so, why does the screen still works?

[rabesocke]

Another hint for finding the real problem: The error only occures when the cover is completely open - any other position don't raise the error in my installation.

If so, why does the screen still works?

The cover don't work via HA. If I change the position with the Somfy App or the remote, HA works again until you move the cover to the position "open".

I think, the position returns a String or Null for "open" and then the cast()-method returns a String. A convert with int() probably returns an integer 0. But that's only a guess.

[springstan]

@rabesocke could you please do some debugging especially what value gets returned when the cover is completely open?

[pascalsaul]

The screen cover does work but not the shutter doesn't through HA.

After the fix from above all do work again:

Current JSON from Somfy API from above image:

[
  {
    "id": "<id>",
    "type": "exterior_blind_positionable_stateful_generic",
    "parent_id": "e9ffe6d4-e2d49a25-d318b13c-08401620",
    "categories": [
      "actuator",
      "exterior_blind"
    ],
    "states": [
      {
        "name": "position",
        "value": 100,
        "type": "integer"
      }
    ],
    "capabilities": [
      {
        "name": "position",
        "parameters": [
          {
            "name": "position",
            "type": "integer"
          }
        ]
      },
      {
        "name": "close",
        "parameters": []
      },
      {
        "name": "identify",
        "parameters": []
      },
      {
        "name": "open",
        "parameters": []
      },
      {
        "name": "stop",
        "parameters": []
      }
    ],
    "site_id": "<site id>",
    "name": "screen_front",
    "available": true
  },
  {
    "id": "<id>",
    "type": "roller_shutter_positionable_stateful_generic",
    "parent_id": "<id>",
    "categories": [
      "actuator",
      "roller_shutter"
    ],
    "states": [
      {
        "name": "position",
        "value": 100,
        "type": "integer"
      }
    ],
    "capabilities": [
      {
        "name": "position",
        "parameters": [
          {
            "name": "position",
            "type": "integer"
          }
        ]
      },
      {
        "name": "close",
        "parameters": []
      },
      {
        "name": "identify",
        "parameters": []
      },
      {
        "name": "open",
        "parameters": []
      },
      {
        "name": "stop",
        "parameters": []
      }
    ],
    "site_id": "<site id>",
    "name": "shutter_back_left",
    "available": true
  },
  {
    "id": "<id>",
    "type": "roller_shutter_positionable_stateful_generic",
    "parent_id": "<id>",
    "categories": [
      "actuator",
      "roller_shutter"
    ],
    "states": [
      {
        "name": "position",
        "value": "-1",
        "type": "integer"
      }
    ],
    "capabilities": [
      {
        "name": "position",
        "parameters": [
          {
            "name": "position",
            "type": "integer"
          }
        ]
      },
      {
        "name": "close",
        "parameters": []
      },
      {
        "name": "identify",
        "parameters": []
      },
      {
        "name": "open",
        "parameters": []
      },
      {
        "name": "stop",
        "parameters": []
      }
    ],
    "site_id": "<site id>",
    "name": "shutter_front_left",
    "available": true
  },
  {
    "id": "<id>",
    "type": "roller_shutter_positionable_stateful_generic",
    "parent_id": "<id>",
    "categories": [
      "actuator",
      "roller_shutter"
    ],
    "states": [
      {
        "name": "position",
        "value": "-1",
        "type": "integer"
      }
    ],
    "capabilities": [
      {
        "name": "position",
        "parameters": [
          {
            "name": "position",
            "type": "integer"
          }
        ]
      },
      {
        "name": "close",
        "parameters": []
      },
      {
        "name": "identify",
        "parameters": []
      },
      {
        "name": "open",
        "parameters": []
      },
      {
        "name": "stop",
        "parameters": []
      }
    ],
    "site_id": "<site id>",
    "name": "shutter_front_right",
    "available": true
  },
  {
    "id": "<id>",
    "type": "roller_shutter_positionable_stateful_generic",
    "parent_id": "<id>",
    "categories": [
      "actuator",
      "roller_shutter"
    ],
    "states": [
      {
        "name": "position",
        "value": "-1",
        "type": "integer"
      }
    ],
    "capabilities": [
      {
        "name": "position",
        "parameters": [
          {
            "name": "position",
            "type": "integer"
          }
        ]
      },
      {
        "name": "close",
        "parameters": []
      },
      {
        "name": "identify",
        "parameters": []
      },
      {
        "name": "open",
        "parameters": []
      },
      {
        "name": "stop",
        "parameters": []
      }
    ],
    "site_id": "<site id>",
    "name": "shutter_back_right",
    "available": true
  },
  {
    "id": "<id>",
    "type": "hub_connexoon",
    "categories": [
      "hub"
    ],
    "states": [],
    "capabilities": [],
    "site_id": "<site id>",
    "name": "CONNEXOON",
    "available": true,
    "version": "2019.5.4-5"
  }
]

[pascalsaul]

So a shutter which is closed it returns "-1"
When it is open you get "100".

So the formula is 100 - -1 right...

I think there should be a case statement when the API returns -1 it is fully closed. This applies for both; the shutters and screen. Only the screen is working "reverse".

[rabesocke]

Yes, I can confirm this.

I added for debugging some outputs in the method:

_LOGGER.warning("print get_position: %s", self.cover.get_position())     
_LOGGER.warning("type get_position: %s", type(self.cover.get_position()))
_LOGGER.warning("type int(get_position): %s", type(int(self.cover.get_position())))

The output is (abstract):

2019-12-19 19:30:39 WARNING (MainThread) [cover] print get_position: 100
2019-12-19 19:30:39 WARNING (MainThread) [cover] type get_position: <class 'int'>
2019-12-19 19:30:39 WARNING (MainThread) [cover] type int(get_position): <class 'int'>

2019-12-19 19:30:40 WARNING (MainThread) [cover] print get_position: -1
2019-12-19 19:30:40 WARNING (MainThread) [cover] type get_position: <class 'str'>
2019-12-19 19:30:40 WARNING (MainThread) [cover] type int(get_position): <class 'int'>

2019-12-19 19:30:39 WARNING (MainThread) [cover] print get_position: 85
2019-12-19 19:30:39 WARNING (MainThread) [cover] type get_position: <class 'int'>
2019-12-19 19:30:39 WARNING (MainThread) [cover] type int(get_position): <class 'int'>

That's the confirmation of my guess. The cast() method returns a String if the cover is "open" and the conversion with int() returns the Integer -1.

There are probably two possible fixes:

• add another conversion / cast as described above: https://github.com/home-assistant/home-assistant/issues/30069#issuecomment-567591557
• change cast() to int() in the pymfy library: https://github.com/tetienne/somfy-open-api/blob/c4b9f32203380f49db6303ca77231f543d303c7b/pymfy/api/devices/roller_shutter.py#L10-L11

[springstan]

From cover/__init__.py:

None is unknown, 0 is closed, 100 is fully open.

@pascalsaul therefore, I completely agree, there should be a case for the return value -1

[rabesocke]

Then was there a change in the API? My cover is open and returns -1.

But as you said, there has to be a solution for -1. With the fix above the Somfy integration returns 101 for fully open.

[rabesocke]

I realized, that my fast fix has one problem. As I wrote, the method returns 101 for the position "open". HA doesn't recognize this value as "open" and displays the cover with both directions "up" and "down" as active.

At the moment I use this fix for the method current_cover_position:

    @property                               
    def current_cover_position(self):
        """Return the current position of cover shutter."""
        position = None
        if self.has_capability("position"):
            position = 100 - int(self.cover.get_position())
        if position > 100:
            position = 100                                          
        return position

[tetienne]

Hi guys,

Sorry for the delay. I can confirm the bug too. It remind my an old one. At the beginning of my work, this issue was already existing. Somfy was returning the wrong position. They returned the position minus one, but it was at least an integer by the past. I will contact them and ask them if they can fix this issue. They already fixed several bugs I reported them.

If they are too lazy, I will have to patch the somfy-open-api: https://github.com/tetienne/somfy-open-api.

[pascalsaul]

Hi guys,

Sorry for the delay. I can confirm the bug too. It remind my an old one. At the beginning of my work, this issue was already existing. Somfy was returning the wrong position. They returned the position minus one, but it was at least an integer by the past. I will contact them and ask them if they can fix this issue. They already fixed several bugs I reported them.

If they are too lazy, I will have to patch the somfy-open-api: https://github.com/tetienne/somfy-open-api.

Can you also ask why they don't support the HorizontalAwning through their API?

Through the old Tahoma integration it was working...

BTW:

        "name": "position",
        "value": "-1",
        "type": "integer"

In the JSON reply from the API it is an integer?

[tetienne]

@pascalsaul The guys which lead the API want to support all the devices. But apparently, this is not yet the first priority...

[rabesocke]

Hi guys,
Sorry for the delay. I can confirm the bug too. It remind my an old one. At the beginning of my work, this issue was already existing. Somfy was returning the wrong position. They returned the position minus one, but it was at least an integer by the past. I will contact them and ask them if they can fix this issue. They already fixed several bugs I reported them.
If they are too lazy, I will have to patch the somfy-open-api: https://github.com/tetienne/somfy-open-api.

Can you also ask why they don't support the HorizontalAwning through their API?

Through the old Tahoma integration it was working...

That would be great 👍

BTW:

        "name": "position",
        "value": "-1",
        "type": "integer"

In the JSON reply from the API it is an integer?

That's right. The Somfy API seems to return -1 as an Integer.
The problem is - as I wrote - the cast() method of the typing library. This method returns a String for -1. You can see this at my debug log output: https://github.com/home-assistant/home-assistant/issues/30069#issuecomment-567609305

[tetienne]

Let's see the Somfy answer. If they fix quickly their API, we will have to do nothing. Else, I will update my library. The fix is pretty easy.

[rabesocke]

Let's see the Somfy answer. If they fix quickly their API, we will have to do nothing. Else, I will update my library. The fix is pretty easy.

What do you expect from Somfy? That they return 0 for the position "open"? If they change it to this value, everything is ok.

But if they still return -1 then we have at the end the value 101 in HA for the position "open" which leads to another problem (https://github.com/home-assistant/home-assistant/issues/30069#issuecomment-567842234)

[tetienne]

That's definitively a bug on their side. The position must be an integer:

 {
        "name": "position",
        "parameters": [
          {
            "name": "position",
            "type": "integer"
          }
        ]
      },

For HA don't worry, I will do the modification within my library to always returned a value between 0 and 100.

[rabesocke]

But it seems to be an Integer right now - see https://github.com/home-assistant/home-assistant/issues/30069#issuecomment-567599279

The problem seems to be the cast() method which returns the value as a String.

[tetienne]

The cast method does not change the returned type: https://docs.python.org/3/library/typing.html#typing.cast
This method is just here to help the type checker.
Somfy always returns an integer for the position, except now for the closed position which is "-1" (the string) instead of 0 (the integer). That's the bug. I will patch the library this week-end, if they don't change this behavior.

[rabesocke]

Sorry, I overlooked the quotation marks in the JSON. That's indeed a bug on Somfy side.

[Dulcow]

It looks like even with HA Cloud integration, the error occurs, right? I just tried to move from direct integration with a custom application to HAC.

[tetienne]

Yes, custom or cloud application is impacted. That's Somfy API behind both.

[tetienne]

I got an answer from the Somfy support. They will fix this quickly!

[Dulcow]

Everything is back to normal. Both integrations are working! :)

[yjajkiew]

I just tried it : API is working as before. I'm still having a "URI mistmatched" using the configuration.yaml so I had to use the Home Assistant Cloud integration.

[tetienne]

@pascalsaul Can you please close the issue?

[rabesocke]

It looks good. The debug outputs:

_LOGGER.warning("get_position: %s", self.cover.get_position())
-> get_position: 0

_LOGGER.warning("type(get_position): %s", type(self.cover.get_position()))
-> type(get_position): <class 'int'>