Core: 0.115 SolarEdge integration is working very unstable

[majkers]

The problem

After upgrading to HA 0.115 from 0.114.4 SolarEdge integration is working very unstable. I get errors in HA logs with info that update fails eventhough when I go to https://monitoringapi.solaredge.com/site/MY_SITE_ID/overview?api_key=MY_KEY i get info without any errors

Environment

I am using HASSIO on RPi3

• Home Assistant Core release with the issue: 0.115
• Last working Home Assistant Core release (if known): 0.114.4
• Operating environment (OS/Container/Supervised/Core): HASSIO
• Integration causing this issue: SolarEdge
• Link to integration documentation on our website: https://www.home-assistant.io/integrations/solaredge/

Problem-relevant configuration.yaml

Done via UI

Traceback/Error logs

2020-09-18 11:28:31 ERROR (MainThread) [homeassistant.components.solaredge.sensor] Could not retrieve details from SolarEdge API
(when booting)

(and the when readding integration)
Logger: homeassistant.helpers.entity
Source: components/solaredge/sensor.py:270
First occurred: 11:38:00 (1 occurrences)
Last logged: 11:38:00
Update for sensor.solaredge_storage_level fails

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 278, in async_update_ha_state
    await self.async_device_update()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 471, in async_device_update
    await self.hass.async_add_executor_job(self.update)  # type: ignore
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/homeassistant/homeassistant/components/solaredge/sensor.py", line 270, in update
    self._state = attr["soc"]
TypeError: 'NoneType' object is not subscriptable

Additional information

[probot-home-assistant[bot]]

solaredge documentation
solaredge source
_{^{(message by IssueLinks)}}

[springstan]

@mhaack could you please take a look at this? Since it seems to come from your additions in #37826

[mhaack]

@majkers can share a few details on your solaredge setup? Do you have a battery connected?

Can you also share a dump of https://monitoringapi.solaredge.com/site/MY_SITE_ID/overview?api_key=MY_KEY?

[marcelveldt]

There's an easy fix for this.
Change line 270 of sensor.py to:

if attr:
    self._state = attr["soc"]

[majkers]

@mhaack No batteries. Just solar inverter with 10 panels. And here is the output:

<overview> <lastUpdateTime>2020-09-19 09:06:44</lastUpdateTime> <lifeTimeData> <energy>532247.0</energy> </lifeTimeData> <lastYearData> <energy>532247.0</energy> </lastYearData> <lastMonthData> <energy>228324.0</energy> </lastMonthData> <lastDayData> <energy>1899.0</energy> </lastDayData> <currentPower> <power>1629.0092</power> </currentPower> <measuredBy>INVERTER</measuredBy> </overview>

[mhaack]

Can you share the result for https://monitoringapi.solaredge.com/site/SITE_ID/currentPowerFlow?api_key=MY_KEY please? This is the API call with the storage data is retrieved from, but only if present there.

[majkers]

@mhaack ok just this in result:

<siteCurrentPowerFlow/>

[mhaack]

? interesting is the inverter switched off? Even with batteries, this should show energy flow from panels to consumption or grid. For me this looks like

<siteCurrentPowerFlow>
  <updateRefreshRate>3</updateRefreshRate>
  <unit>kW</unit>
  <GRID>
    <status>Active</status>
    <currentPower>0.02</currentPower>
  </GRID>
  <LOAD>
    <status>Active</status>
    <currentPower>3.47</currentPower>
  </LOAD>
  <PV>
    <status>Active</status>
    <currentPower>4.53</currentPower>
  </PV>
  <STORAGE>
    <status>Charging</status>
    <currentPower>1.08</currentPower>
    <chargeLevel>44</chargeLevel>
    <critical>false</critical>
  </STORAGE>
  <connections>
    <connection>
      <from>GRID</from>
      <to>Load</to>
    </connection>
    <connection>
      <from>PV</from>
      <to>Load</to>
    </connection>
    <connection>
      <from>PV</from>
      <to>Storage</to>
    </connection>
  </connections>
</siteCurrentPowerFlow>

My expectation would be yours look similar except the STORAGE block.

[marcelveldt]

@mhaack sorry to chumb in but you're missing the obvious here. The api is not available from time to time in which case the result is empty. The code is guarded for that by using .get on the dict objects execpt for this block. You're trying to use the data which is None at that moment. Just add another guard like I suggested and you're all set.

Alternative solution is find out why the api is empty sometimes... Guess it's a bit overloaded.

[mhaack]

@marcelveldt thanks yes I'm about to do add the additioninal guard as you suggested. Just want to understand the different data structures for various inverster setups look like.

[majkers]

I guess it depends of inverter or something. While I keep getting nothing in the last one I keep getting something in the previous one. Checking the log, this API seems to be unavailable very often casue this message occurs over 400 times nad I have my installation just for over a month now. Maybe You could add some kind of timestamp of last update so one could check when was the last proper update?

There is even something in that XML:

eg. <lastUpdateTime>2020-09-19 12:45:59</lastUpdateTime>

[majkers]

One more thing. According to https://www.solaredge.com/sites/default/files/se_monitoring_api.pdf this API call with currentPowerFlow "Applies when export , importand consumption can be measured"...And in my installation it can't

[mhaack]

Yes it depends on the inverter configuration, as you wrote. So I assume also other power flow sensors are also not available in your case. To you have for example sensor.solaredge_exported_power in your HA setup? Does this sensor exit? If yes I guess it always stays 0.

With #40295 I already created a fix for this problem.

[majkers]

I lied it is not 400 but 300. I'm not sure that it is so buggy...

exported_power stays unknown

[mhaack]

HA tries to update the sensor in a loop, that's why you see that so often. Based on the code I think it should at least have no influence on the other sensors.

With the PR fix the error should disappear.

[majkers]

@mhaack I am still getting this:

Logger: homeassistant.helpers.entity
Source: components/solaredge/sensor.py:442
First occurred: 26 września 2020, 0:07:52 (4477 occurrences)
Last logged: 2:47:17

Update for sensor.solaredge_imported_power fails
Update for sensor.solaredge_production_power fails
Update for sensor.solaredge_consumption_power fails
Update for sensor.solaredge_selfconsumption_power fails
Update for sensor.solaredge_exported_power fails
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 278, in async_update_ha_state
    await self.async_device_update()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 471, in async_device_update
    await self.hass.async_add_executor_job(self.update)  # type: ignore
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/homeassistant/homeassistant/components/solaredge/sensor.py", line 217, in update
    self.data_service.update()
  File "/usr/src/homeassistant/homeassistant/util/__init__.py", line 239, in wrapper
    result = method(*args, **kwargs)
  File "/usr/src/homeassistant/homeassistant/components/solaredge/sensor.py", line 442, in update
    self.data[energy_type] = row["value"]
KeyError: 'value'

[mhaack]

This one is coming from another change (https://github.com/home-assistant/core/pull/34525) I can try look into this, error is from different sensor https://github.com/home-assistant/core/blob/dev/homeassistant/components/solaredge/sensor.py#L442

[majkers]

Thanks. If you need seperate issue for this, let me know.

[majkers]

Also this did not disappear....

Logger: homeassistant.helpers.entity
Source: components/solaredge/sensor.py:195
First occurred: 13:32:52 (1145 occurrences)
Last logged: 15:26:56

Update for sensor.solaredge_meters fails
Update for sensor.solaredge_sensors fails
Update for sensor.solaredge_gateways fails
Update for sensor.solaredge_batteries fails
Update for sensor.solaredge_inverters fails
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 278, in async_update_ha_state
    await self.async_device_update()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 471, in async_device_update
    await self.hass.async_add_executor_job(self.update)  # type: ignore
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/homeassistant/homeassistant/components/solaredge/sensor.py", line 195, in update
    self._state = self.data_service.data[self._json_key]
KeyError: 'meters'

[majkers]

@mhaack I decided to create new issue #40780

[mhaack]

+1 I'm not familiar with the areas of the code anyway. However, I can have a look as well. The challenge for me is I can not reproduce the issues with my inverter setup and have only this one site/api_key to test with. Maybe, if we find a secure way, you could share your site id and I could have a look.