Core: Enphase Envoy broke in 0.96.2

[PaulAntonDeen]

Home Assistant release with the issue:

0.96.2

Last working Home Assistant release (if known):
0.95.x

Operating environment (Hass.io/Docker/Windows/etc.):

Hass.io

Component/platform:

Enphase-envoy

Description of problem:
Envoy sensors do not have any value

Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):

  - platform: enphase_envoy
    ip_address: 192.168.2.7
    monitored_conditions:
      - production
      - daily_production
      - seven_days_production
      - lifetime_production

Traceback (if applicable):

Sat Jul 20 2019 08:09:27 GMT+0200 (Central European Summer Time)
Error while setting up platform enphase_envoy
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 126, in _async_setup_platform
    SLOW_SETUP_MAX_WAIT)
  File "/usr/local/lib/python3.7/asyncio/tasks.py", line 442, in wait_for
    return fut.result()
  File "/usr/src/homeassistant/homeassistant/components/enphase_envoy/sensor.py", line 45, in async_setup_platform
    from envoy_reader.envoy_reader import EnvoyReader
  File "/usr/local/lib/python3.7/site-packages/envoy_reader/envoy_reader.py", line 8, in <module>
    import requests_async as requests
ModuleNotFoundError: No module named 'requests_async'

Additional information:

[jesserizzo]

For some reason it isn't installing the dependencies that it needs. If you can get to a command line where ever your home assistant lives, you can just run
sudo pip3 install requests_async
to get that dependency manually. I'm planning to fix this for the next HA release.

[frenck]

@jeanregisser That command will not work on Hass.io.

[scottyphillips]

@frenck you can get access to Docker on hass.io using the SSH add-one and then you should be able to run docker exec -it homeasssitant /bin/bashand from there run pip3 install requests_async

[PaulAntonDeen]

@frenck you can get access to Docker on hass.io using the SSH add-one and then you should be able to run docker exec -it homeasssitant /bin/bashand from there run pip3 install requests_async

Not through the SSH addon, but you can get there through the debugger I found. Follow the debugger instructions to get SSH access, login, and then the commands as mentioned by Scotty.
Thanks, I was able to execute, that fixed the problem for now.

[PaulAntonDeen]

Seems it doesn't solve everything: This shows up in my Developer Tools - Info log. Got a response from 'http://192.168.2.7/production.json', but metric could not be found. Maybe your model of Envoy doesn't support the requested metric. So either it is not correctly parsing the JSON or the JSON file on the envoy changed at the same time that HA updated?

The contents of the referenced JSON: {"production":[{"type":"inverters","activeCount":12,"readingTime":1563911519,"wNow":-2,"whLifetime":1870445}],"storage":[{"type":"acb","activeCount":0,"readingTime":0,"wNow":0,"whNow":0,"state":"idle"}]}

[scottyphillips]

@frenck you can get access to Docker on hass.io using the SSH add-one and then you should be able to run docker exec -it homeasssitant /bin/bashand from there run pip3 install requests_async

Not through the SSH addon, but you can get there through the debugger I found. Follow the debugger instructions to get SSH access, login, and then the commands as mentioned by Scotty.
Thanks, I was able to execute, that fixed the problem for now.

SSH is via an installable community add-on that you can configure via the HASSIO plug-in menu. If you turn protection mode off you can get access to Docker.

https://github.com/hassio-addons/addon-ssh

[PaulAntonDeen]

Just downgraded to 0.95.4 and envoy is fully working again including 4 sensors Current production, todays, last seven days and lifetime.

[watson81]

For better or worse, the SSH add-on does not provide easy access to docker, even with Protection Mode disabled. That's because every addon is loaded in an independent Docker container and has a limited set of directories mounted into the container. I.e. the addon_core_ssh container doesn't mount the docker socket or the homeassistant container's directory containing the Python environment.
However, it is relatively easy to avoid this and apply the workaround:

1. Follow the instructions here to configure HassOS to allow you to SSH into the parent system.
   Note: The parent system's SSH server is rather simple. It doesn't support ed25519 keys. I can only vouch for RSA keys.
   Warning: This opens your system to attack if you don't protect it properly. Make sure to use a strong SSH key. Don't connect port 22222 to the Internet. Consider disabling ssh after you're done with this.
2. SSH into the parent system: ssh -p 22222 [email protected]
3. You'll be in the Hass.io command line. Run the login command to gain shell access to the parent OS.
4. Run docker exec -it homeassistant pip3 install requests_async
5. Exit the parent OS: exit
6. Restart the Home Assistant server via the WebUI.

Note: this will be undone when Home Assistant is upgraded, assuming that this ticket hasn't been fixed by then. So, you'll need to go through steps 2-6 again.

[Dudu67]

Please try also to correct #21682 too.
Thanks.

[Siggy101]

Is there any progress on this fix? It's been a couple of weeks and no change despite 2 more versions being released.

[gaetanars]

Dear all,

I try to propose a PR for this missing dependency (#25626).
In the meantime I use this in my docker-compose file :
command: bash -c "pip3 install requests_async && python -m homeassistant --config /config"

[jesserizzo]

Hey, sorry I've been pretty absent from this. The original issue that @PaulAntonDeen referenced I have fixed, but my PR failed the CI/CD checks due to a dependency not working with python 3.5. Well last week was the last HA release to support python 3.5 anyway so next week I assume the PR will pass CI/CD.

The issue that @Dudu67 mentioned I hadn't seen before. I'll take a look into that as well.

[Siggy101]

Hey, sorry I've been pretty absent from this. The original issue that @PaulAntonDeen referenced I have fixed, but my PR failed the CI/CD checks due to a dependency not working with python 3.5. Well last week was the last HA release to support python 3.5 anyway so next week I assume the PR will pass CI/CD.

The issue that @Dudu67 mentioned I hadn't seen before. I'll take a look into that as well.

Many thanks for your work on this. It's greatly appreciated.
Does your reply mean that we should be seeing a fix for the Envoys not showing any data in a release next week?

[jesserizzo]

@Siggy101 Yep, my PR just got merged into Home Assistant today.

[PaulAntonDeen]

Great, i'll test it when the new HA version comes out! Thanks for your work!

[ryannazaretian]

Something is still off:

Traceback (most recent call last):
  File "/usr/src/app/homeassistant/helpers/entity.py", line 249, in async_update_ha_state
    await self.async_device_update()
  File "/usr/src/app/homeassistant/helpers/entity.py", line 417, in async_device_update
    await self.async_update()
  File "/usr/src/app/homeassistant/components/enphase_envoy/sensor.py", line 126, in async_update
    inverters = await (EnvoyReader(self._ip_address).inverters_production())
  File "/usr/local/lib/python3.7/site-packages/envoy_reader/envoy_reader.py", line 332, in inverters_production
    await self.get_serial_number()
  File "/usr/local/lib/python3.7/site-packages/envoy_reader/envoy_reader.py", line 71, in get_serial_number
    timeout=30, allow_redirects=False)
  File "/usr/local/lib/python3.7/site-packages/requests_async/api.py", line 11, in get
    return await request("get", url, params=params, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests_async/api.py", line 6, in request
    return await session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests_async/sessions.py", line 79, in request
    resp = await self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests_async/sessions.py", line 136, in send
    r = await adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/requests_async/adapters.py", line 62, in send
    raise ReadTimeout(err, request=request)
requests.exceptions.ReadTimeout

[jesserizzo]

@ryannazaretian Sometimes my envoy just stops responding for a while. Give it a few hours, maybe reboot HA and see if that goes away.

[DavidDeSloovere]

The fix worked for me.

[Siggy101]

@Siggy101 Yep, my PR just got merged into Home Assistant today.

Many thanks for your awesome work. The fix has worked perfectly for me once I installed v0.97.0
One thing to note is the the config checker threw a gajillion errors. I imagine this is because the prerequisites are obviously not installed yet.
I bit the bullet and installed and it went smoothly. Hassio was extremely sluggish for about 2 hours after the install. Perhaps it was busy tidying stuff up.....

[ryannazaretian]

2 days later, still seeing the Read Timeouts. I've reinstalled the Home Assistant docker and restarted the controller outside. It seems to be poking the Enphase controller very frequently.

[jesserizzo]

@ryannazaretian Can you go to your Envoy's IP address in a web browser? Do the stack traces for the read timeouts always have this line in them?

File "/usr/src/app/homeassistant/components/enphase_envoy/sensor.py", line 126, in async_update
inverters = await (EnvoyReader(self._ip_address).inverters_production())

If yes to both, then try taking out the - inverters line in your config and see if that works.

[ryannazaretian]

I ended up switching to the Rest API to read the JSON directly. I actually
prefer the Rest method since it gives me more information and should have
better reliability.

On Sat, Aug 10, 2019 at 8:05 AM Jesse Rizzo [e-mail]
wrote:

@ryannazaretian https://github.com/ryannazaretian Can you go to your
Envoy's IP address in a web browser? Do the stack traces for the read
timeouts always have this line in them?

File "/usr/src/app/homeassistant/components/enphase_envoy/sensor.py", line
126, in async_update
inverters = await (EnvoyReader(self._ip_address).inverters_production())

If yes to both, then try taking out the - inverters line in your config
and see if that works.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/home-assistant/home-assistant/issues/25331?email_source=notifications&email_token=AALQG7SWIKUOXSDDB65UJ4DQD24K7A5CNFSM4IFNXRXKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4ANERA#issuecomment-520147524,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AALQG7T474WOZ6EAEOTWPWLQD24K7ANCNFSM4IFNXRXA
.

>

Ryan Nazaretian

[PaulAntonDeen]

Finally did a fresh install of hassio & HA 0.97.2, it does fix the async problem but now I still get this error in the log and no production information:

Got a response from 'http://192.168.2.7/production.json', but metric could not be found. Maybe your model of Envoy doesn't support the requested metric.
11:21 components/enphase_envoy/sensor.py (ERROR) - message first occured at 10:26 and shows up 428 times

[jesserizzo]

@PaulAntonDeen And to be clear, it was working previously? What version of the Envoy do you have? What parameters are you trying to monitor? If you remove some of them from your config does the message go away?

[PaulAntonDeen]

@PaulAntonDeen And to be clear, it was working previously? What version of the Envoy do you have? What parameters are you trying to monitor? If you remove some of them from your config does the message go away?

Parameters as in the start post:
- platform: enphase_envoy ip_address: 192.168.2.7 monitored_conditions: - production - daily_production - seven_days_production - lifetime_production

If I swap the SD card in my rPi to the other one with 0.95.4 it works perfectly fine.

I also wrote the contents of the referenced json file in post 5 (26 days ago)

Thanks for your help!
Paul

[jesserizzo]

@PaulAntonDeen Ok, I think I got it. If you're willing and able to test something for me that would be cool. If you can drop this file in where ever your python libraries live (you can search for envoy_reader) that should fix it. https://github.com/jesserizzo/envoy_reader/blob/fix_envoy_detection/envoy_reader/envoy_reader.py

[PaulAntonDeen]

Could it be that it is reading the wrong file?
http://envoy/production.json contains
{"production":[{"type":"inverters","activeCount":12,"readingTime":0,"wNow":0,"whLifetime":2232467}],"storage":[{"type":"acb","activeCount":0,"readingTime":0,"wNow":0,"whNow":0,"state":"idle"}]}

http://envoy/api/v1/production contains the much more useful:
{
"wattHoursToday": 0,
"wattHoursSevenDays": 74387,
"wattHoursLifetime": 2232467,
"wattsNow": 0
}

[PaulAntonDeen]

@PaulAntonDeen And to be clear, it was working previously? What version of the Envoy do you have? What parameters are you trying to monitor? If you remove some of them from your config does the message go away?

Envoy-S-Standard-EU
Part Number
800-00553-r02
Software Version
R4.10.35 (6ed292)

Monitoring 4 production parameters, nothing else. Reducing them does not make the message go away.

[PaulAntonDeen]

As an alternative, this works using the rest api:

  - platform: rest
    name: solar production
    json_attributes:
      - wattHoursToday
      - wattHoursSevenDays
      - wattHoursLifetime
      - wattsNow
    resource: http://192.168.2.7/api/v1/production
  - platform: template
    sensors:
      solar_now_kw:
        friendly_name: "Solar current production"
        value_template: "{{ (states.sensor.solar_production.attributes['wattsNow'] | float / 1000 ) | round(3) }}"
        unit_of_measurement: 'kW'
      solar_today_kwh:
        friendly_name: "Solar Today"
        value_template: "{{ (states.sensor.solar_production.attributes['wattHoursToday'] | float / 1000 ) | round(1) }}"
        unit_of_measurement: 'kWh'
      solar_week_kwh:
        friendly_name: "Solar last 7 days"
        value_template: "{{ (states.sensor.solar_production.attributes['wattHoursSevenDays'] | float / 1000 ) | round(1) }}"
        unit_of_measurement: 'kWh'
      solar_lifetime_mwh:
        friendly_name: "Solar Lifetime"
        value_template: "{{ (states.sensor.solar_production.attributes['wattHoursLifetime'] | float / 1000000 ) | round(2) }}"
        unit_of_measurement: 'mWh'

[PaulAntonDeen]

Ok, I managed to get to the file and replace it: I get this error in the log:

Log Details (ERROR)
Mon Aug 19 2019 19:28:38 GMT+0200 (Central European Summer Time)
Error while setting up platform enphase_envoy
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 149, in _async_setup_platform
    await asyncio.wait_for(asyncio.shield(task), SLOW_SETUP_MAX_WAIT)
  File "/usr/local/lib/python3.7/asyncio/tasks.py", line 442, in wait_for
    return fut.result()
  File "/usr/src/homeassistant/homeassistant/components/enphase_envoy/sensor.py", line 53, in async_setup_platform
    from envoy_reader.envoy_reader import EnvoyReader
  File "/usr/local/lib/python3.7/site-packages/envoy_reader/envoy_reader.py", line 7
    <!DOCTYPE html>
    ^
SyntaxError: invalid syntax

[PaulAntonDeen]

I ended up switching to the Rest API to read the JSON directly. I actually prefer the Rest method since it gives me more information and should have better reliability.
On Sat, Aug 10, 2019 at 8:05 AM Jesse Rizzo @.*> wrote: @ryannazaretian https://github.com/ryannazaretian Can you go to your Envoy's IP address in a web browser? Do the stack traces for the read timeouts always have this line in them? File "/usr/src/app/homeassistant/components/enphase_envoy/sensor.py", line 126, in async_update inverters = await (EnvoyReader(self._ip_address).inverters_production()) If yes to both, then try taking out the - inverters line in your config and see if that works. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#25331?email_source=notifications&email_token=AALQG7SWIKUOXSDDB65UJ4DQD24K7A5CNFSM4IFNXRXKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4ANERA#issuecomment-520147524>, or mute the thread https://github.com/notifications/unsubscribe-auth/AALQG7T474WOZ6EAEOTWPWLQD24K7ANCNFSM4IFNXRXA .
-- Ryan Nazaretian

By default that only updates every 5 or 6 minutes. Have you found how to refresh more often?

[PaulAntonDeen]

@jesserizzo did you get a chance to look at the code error I got when I replaced the default .py with the one you linked? See 2 posts back for the errors and 2 post before that for Envoy version information.

[jesserizzo]

@PaulAntonDeen Can you try running the python script I linked again? If it's easier you can just run it on the command line instead of dropping it into your HA install.

There's no logical reason for that to not work based on the json replies to the endpoints that you posted above. I'm hoping that last time you tried to run it your Envoy was just not responding or something.

[PaulAntonDeen]

Sure, will do when I get back home (travelling for work this week).

[PaulAntonDeen]

@PaulAntonDeen Can you try running the python script I linked again? If it's easier you can just run it on the command line instead of dropping it into your HA install.

There's no logical reason for that to not work based on the json replies to the endpoints that you posted above. I'm hoping that last time you tried to run it your Envoy was just not responding or something.

I tried to do it but in Hassio it takes a bit of work to get to the right docker container and I forgot how to do it :( I see this change is likely coming in the next update 0.99.3? so I might just wait for that and open an issue if it doesn't work. Sound ok to you?