Core: 113.x Tuya : Update for switch.tuya fails : 'Connection aborted.', RemoteDisconnected('Remote end closed connection without response'

[pergolafabio]

The problem

I have Tuya ingegration (Smart Life)
All is working fine, but i see a lot of errors in my log files... its not constant though, i just see them in my log file, sometimes afer an an hour

Environment

• Home Assistant Core release with the issue: 113.0
• Last working Home Assistant Core release (if known): not sure
• Operating environment (OS/Container/Supervised/Core): HassOS 4.11
• Integration causing this issue: Tuya (Smart Life)

Problem-relevant configuration.yaml

No Yaml, used integration

Traceback/Error logs

2020-07-24 12:49:04 ERROR (MainThread) [homeassistant.helpers.entity] Update for switch.tuya_2 fails
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 670, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 426, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 421, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/lib/python3.8/http/client.py", line 1332, in getresponse
    response.begin()
  File "/usr/local/lib/python3.8/http/client.py", line 303, in begin
    version, status, reason = self._read_status()
  File "/usr/local/lib/python3.8/http/client.py", line 272, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 726, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/site-packages/urllib3/util/retry.py", line 403, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.8/site-packages/urllib3/packages/six.py", line 734, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 670, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 426, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 421, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/lib/python3.8/http/client.py", line 1332, in getresponse
    response.begin()
  File "/usr/local/lib/python3.8/http/client.py", line 303, in begin
    version, status, reason = self._read_status()
  File "/usr/local/lib/python3.8/http/client.py", line 272, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 272, in async_update_ha_state
    await self.async_device_update()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 465, in async_device_update
    await self.hass.async_add_executor_job(
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/homeassistant/homeassistant/components/tuya/__init__.py", line 254, in update
    self._tuya.update()
  File "/usr/local/lib/python3.8/site-packages/tuyaha/devices/switch.py", line 29, in update
    devices = self.api.discovery()
  File "/usr/local/lib/python3.8/site-packages/tuyaha/tuyaapi.py", line 117, in discovery
    response = self._request("Discovery", "discovery")
  File "/usr/local/lib/python3.8/site-packages/tuyaha/tuyaapi.py", line 163, in _request
    response = self.requestSession.post(
  File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 578, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

[probot-home-assistant[bot]]

Hey there @ollo69, mind taking a look at this issue as its been labeled with an integration (tuya) you are listed as a codeowner for? Thanks!
_{^{(message by CodeOwnersMention)}}

[matisaul]

Same problem here... From Tuya app when I turn off the switch manually, update en 1 second, but in HA delay is between 30-80 seconds.

[jcalderon71]

Same issue for me - Home Assistant 0.114.4

[pezik1]

I have the same problem - Home Assistant 0.114.4
"platform": "smart_life",
"country_code": "420",

but for climate sensor

[oopptt]

I think I have same problem. Here's my log, continously showed since any of the last updates:

Traceback (most recent call last):
File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/helpers/entity.py", line 263, in async_update_ha_state
await self.async_device_update()
File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/helpers/entity.py", line 457, in async_device_update
self.update # type: ignore
File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(self.args, *self.kwargs)
File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/components/tuya/__init__.py", line 254, in update
self._tuya.update()
File "/srv/homeassistant/lib/python3.7/site-packages/tuyaha/devices/switch.py", line 29, in update
devices = self.api.discovery()
File "/srv/homeassistant/lib/python3.7/site-packages/tuyaha/tuyaapi.py", line 117, in discovery
response = self._request("Discovery", "discovery")
File "/srv/homeassistant/lib/python3.7/site-packages/tuyaha/tuyaapi.py", line 164, in _request
(TUYACLOUDURL + "/homeassistant/skill").format(SESSION.region), json=data
File "/srv/homeassistant/lib/python3.7/site-packages/requests/sessions.py", line 578, in post
return self.request('POST', url, data=data, json=json, *kwargs)
File "/srv/homeassistant/lib/python3.7/site-packages/requests/sessions.py", line 530, in request
resp = self.send(prep, *send_kwargs)
File "/srv/homeassistant/lib/python3.7/site-packages/requests/sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "/srv/homeassistant/lib/python3.7/site-packages/requests/adapters.py", line 498, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

Hope somebody could help on this. I'm using 0.114.4

[bdraco]

It looks like the upstream module sleeps for 0.5s to hope that the update has happened.

https://github.com/PaulAnnekov/tuyaha/blob/master/tuyaha/devices/base.py#L41

If it doesn't happen in 0.5s you don't get the update until the next scheduled polling.

.. and there is no retry logic.

.. and it looks like the more device you have, the more the api is going to get overloaded as polling isn't grouped.

[ollo69]

I created a PR 2 months ago in TuyaHA library to group updates in a single call, but for the moment was ignored.
I suggest to try Tuya_Custom component for the moment that include this fix waiting for that PR to be eventually merged.

[bdraco]

@PaulAnnekov Can you help with this issue?

[ollo69]

@bdraco ,

can I ask you an opinion?
At this moment we have a lot of Tuya issues that are connected to some limitation introduced in Tuya API used by TuyaHA library. I created the custom_component Tuya_Custom to make some quick test, but at the end any modification cannot really solve all problems (for instance, now API seems not accept a polling interval below 300 sec - 5 min., really unreliable).
Do you think that make sense go ahead with this integration? May be some HA "big boss" can ask for additional clarification to Tuya, at least to have a clear position about the service provided by them?
I think that continue to officially support a service that cannot be really used is not good also for Home Assistant reputation.

[balloob]

I've pinged them again. I tried that before but no response :/ The original person at Tuya that implemented it left.

[ollo69]

Let me just summarize the current situation based on test result done in "Tuya Custom" component:

• the Tuya dedicated API allow 2 method to query the device status: "query" and "discovery". The first one need a parameter that is the device ID to query, so to have the status for multiple device multiple call must be issued. This is the current method used by the Tuya integration. "Discovery" return the status of all the device in a single call, so should be the preferred method and is the one that I implemented in Tuya Custom.
• Probably due to the very high quantity of requests issued to the cloud (immagine HA users with 20 or more devices that every 30 second make 20 requests), Tuya choose to slow down the availability. At this moment, based on error message returned by the API method, "query" command can be called once every 1 minute (at all, not for device) so is unusable if you have more than 1 device. "Discovery" command can be called once every 5 minutes, this option is usable but refresh time is too high to be used in a reliable way.
• Also consider that both "Discovery" and "Query" do not return all devices property (especially for climate and light entity many information are missing or wrong and not homogeneous) causing a not complete monitoring of the devices status.
• I also try to understand if should be possible to use the standard Tuya API, but this doesn't seems a valid approach because is more dedicated to company that want to publish a custom app using a dedicated SDK. But may be there are other approach that I do not understood.
• As final point, I have to say that @PaulAnnekov, that is the maintainer of TuyaHA library, is non active anymore in the library repository and so I'm not able to bring back my correction implemented in Tuya Custom to the default HA component, as long as it makes sense to do so.

Any additional consideration or suggestion are welcome.

[balloob]

Solution: invest in devices that work with a local API 👍

[aryef]

Let me just summarize the current situation based on test result done in "Tuya Custom" component:

• the Tuya dedicated API allow 2 method to query the device status: "query" and "discovery". The first one need a parameter that is the device ID to query, so to have the status for multiple device multiple call must be issued. This is the current method used by the Tuya integration. "Discovery" return the status of all the device in a single call, so should be the preferred method and is the one that I implemented in Tuya Custom.
• Probably due to the very high quantity of requests issued to the cloud (immagine HA users with 20 or more devices that every 30 second make 20 requests), Tuya choose to slow down the availability. At this moment, based on error message returned by the API method, "query" command can be called once every 1 minute (at all, not for device) so is unusable if you have more than 1 device. "Discovery" command can be called once every 5 minutes, this option is usable but refresh time is too high to be used in a reliable way.
• Also consider that both "Discovery" and "Query" do not return all devices property (especially for climate and light entity many information are missing or wrong and not homogeneous) causing a not complete monitoring of the devices status.
• I also try to understand if should be possible to use the standard Tuya API, but this doesn't seems a valid approach because is more dedicated to company that want to publish a custom app using a dedicated SDK. But may be there are other approach that I do not understood.
• As final point, I have to say that @PaulAnnekov, that is the maintainer of TuyaHA library, is non active anymore in the library repository and so I'm not able to bring back my correction implemented in Tuya Custom to the default HA component, as long as it makes sense to do so.

Any additional consideration or suggestion are welcome.

Hey, I have been using "tuya custom" for two months already, with around 30 devices and 70 automatons. It works very reliably. Thank you, @ollo69 !