Core: Excessive DNS lookups from Tuya integration (2)
The problem
Home Assistant server (v4.10) is making huge numbers of repeated DNS requests for px1.tuyaeu.com.
Every few seconds, it will make several A and an AAAA requests of Domain px1.tuyaeu.com, and this is repeated every 30 seconds or so.
Clean install of Home Assistant (v4.10) image onto new Raspberry Pi 4B (2Gb). Clean startup of Home Assistant with only TUYA integration added (country code: 44). The intergration works allowing switches to be controlled, but it immediately starts flooding the network with DNS lookups for px1.tuyaeu.com
Environment
Host system
Hostname: homeassistant
System: HassOS 4.10
Deployment: production
- Home Assistant Core release with the issue: 4.10
- Last working Home Assistant Core release (if known): not known
- Operating environment (Home Assistant/Supervised/Docker/venv): Fresh install of v4.10 image on Raspberry Pi 4B (2Gb)
- Integration causing this issue: TUYA
- Link to integration documentation on our website:
Problem-relevant configuration.yaml
Default configuration.yaml. No custom changes were made to it whatsoever.
# Configure a default setup of Home Assistant (frontend, api, etc)
default_config:
# Text to speech
tts:
- platform: google_translate
group: !include groups.yaml
automation: !include automations.yaml
script: !include scripts.yaml
scene: !include scenes.yaml
Traceback/Error logs
No obvious errors. The integration works as expected BUT floods Pi-hole with 1000's of DNS lookups to px1.tuyaeu.com
Additional information
Same issue as #26855 which was prematurely closed unresolved.
Extract from Pi-hole log:
Time Type Domain Client Status Reply Action
2020-06-12 10:34:44 A px1.tuyaeu.com pi4b.lan OK (cached) IP (3.3ms)
2020-06-12 10:34:44 A px1.tuyaeu.com pi4b.lan OK (cached) IP (3.3ms)
2020-06-12 10:34:44 AAAA px1.tuyaeu.com pi4b.lan OK (cached) NODATA (3.2ms)
2020-06-12 10:34:44 A px1.tuyaeu.com pi4b.lan OK (cached) IP (3.9ms)
2020-06-12 10:34:44 AAAA px1.tuyaeu.com pi4b.lan OK (cached) NODATA (3.1ms)
2020-06-12 10:34:13 A px1.tuyaeu.com pi4b.lan OK (forwarded) IP (69.3ms)
2020-06-12 10:34:13 A px1.tuyaeu.com pi4b.lan OK (forwarded) IP (113.2ms)
2020-06-12 10:34:13 A px1.tuyaeu.com pi4b.lan OK (forwarded) IP (55.6ms)
2020-06-12 10:34:13 A px1.tuyaeu.com pi4b.lan OK (forwarded) IP (119.8ms)
2020-06-12 10:34:13 A px1.tuyaeu.com pi4b.lan OK (forwarded) IP (91.2ms)
2020-06-12 10:34:13 A px1.tuyaeu.com pi4b.lan OK (forwarded) IP (177.4ms)
2020-06-12 10:34:13 A px1.tuyaeu.com pi4b.lan OK (forwarded) IP (132.8ms)
2020-06-12 10:34:13 A px1.tuyaeu.com pi4b.lan OK (forwarded) IP (136.8ms)
2020-06-12 10:34:13 A px1.tuyaeu.com pi4b.lan OK (forwarded) IP (114.2ms)
2020-06-12 10:34:13 AAAA px1.tuyaeu.com pi4b.lan OK (cached) NODATA (4.2ms)
2020-06-12 10:34:13 A px1.tuyaeu.com pi4b.lan OK (forwarded) IP (119.7ms)
2020-06-12 10:34:13 AAAA px1.tuyaeu.com pi4b.lan OK (cached) NODATA (3.0ms)
2020-06-12 10:33:42 AAAA px1.tuyaeu.com pi4b.lan OK (cached) NODATA (3.1ms)
2020-06-12 10:33:42 AAAA px1.tuyaeu.com pi4b.lan OK (cached) NODATA (3.0ms)
2020-06-12 10:33:42 AAAA px1.tuyaeu.com pi4b.lan OK (cached) NODATA (3.1ms)
2020-06-12 10:33:42 A px1.tuyaeu.com pi4b.lan OK (cached) IP (3.1ms)
2020-06-12 10:33:42 AAAA px1.tuyaeu.com pi4b.lan OK (cached) NODATA (3.1ms)
2020-06-12 10:33:42 AAAA px1.tuyaeu.com pi4b.lan OK (cached) NODATA (3.1ms)
2020-06-12 10:33:42 A px1.tuyaeu.com pi4b.lan OK (cached) IP (3.2ms)
2020-06-12 10:33:42 A px1.tuyaeu.com pi4b.lan OK (cached) IP (3.1ms)
smithbill17
All 24 comments
Hey there @ollo69, mind taking a look at this issue as its been labeled with a integration (tuya) you are listed as a codeowner for? Thanks!
(message by CodeOwnersMention)
![probot-home-assistant[bot] picture](https://avatars2.githubusercontent.com/in/36093?v=4&s=40)
probot-home-assistant[bot]
on 12 Jun 2020
would be this the root cause of this exception?
Jun 15 01:43:55 raspberrypi hass[4833]: urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='px1.tuyaus.com', port=443): Max retries exceeded with url: /homeassistant/skill (
Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0xa0314f70>: Failed to establish a new connection: [Errno 110] Connection timed out'))
Jun 15 01:43:55 raspberrypi hass[4833]: During handling of the above exception, another exception occurred:
Jun 15 01:43:55 raspberrypi hass[4833]: Traceback (most recent call last):
Jun 15 01:43:55 raspberrypi hass[4833]: File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/helpers/entity.py", line 279, in async_update_ha_state
Jun 15 01:43:55 raspberrypi hass[4833]: await self.async_device_update()
Jun 15 01:43:55 raspberrypi hass[4833]: File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/helpers/entity.py", line 472, in async_device_update
Jun 15 01:43:55 raspberrypi hass[4833]: await self.hass.async_add_executor_job(self.update)
Jun 15 01:43:55 raspberrypi hass[4833]: File "/usr/lib/python3.7/concurrent/futures/thread.py", line 57, in run
Jun 15 01:43:55 raspberrypi hass[4833]: result = self.fn(*self.args, **self.kwargs)
Jun 15 01:43:55 raspberrypi hass[4833]: File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/components/tuya/__init__.py", line 254, in update
Jun 15 01:43:55 raspberrypi hass[4833]: self._tuya.update()
Jun 15 01:43:55 raspberrypi hass[4833]: File "/srv/homeassistant/lib/python3.7/site-packages/tuyaha/devices/switch.py", line 23, in update
Jun 15 01:43:55 raspberrypi hass[4833]: devices = self.api.discovery()
Jun 15 01:43:55 raspberrypi hass[4833]: File "/srv/homeassistant/lib/python3.7/site-packages/tuyaha/tuyaapi.py", line 115, in discovery
Jun 15 01:43:55 raspberrypi hass[4833]: response = self._request("Discovery", "discovery")
Jun 15 01:43:55 raspberrypi hass[4833]: File "/srv/homeassistant/lib/python3.7/site-packages/tuyaha/tuyaapi.py", line 161, in _request
Jun 15 01:43:55 raspberrypi hass[4833]: (TUYACLOUDURL + "/homeassistant/skill").format(SESSION.region), json=data
Jun 15 01:43:55 raspberrypi hass[4833]: File "/srv/homeassistant/lib/python3.7/site-packages/requests/api.py", line 119, in post
Jun 15 01:43:55 raspberrypi hass[4833]: return request('post', url, data=data, json=json, **kwargs)
Jun 15 01:43:55 raspberrypi hass[4833]: File "/srv/homeassistant/lib/python3.7/site-packages/requests/api.py", line 61, in request
Jun 15 01:43:55 raspberrypi hass[4833]: return session.request(method=method, url=url, **kwargs)
Jun 15 01:43:55 raspberrypi hass[4833]: File "/srv/homeassistant/lib/python3.7/site-packages/requests/sessions.py", line 530, in request
Jun 15 01:43:55 raspberrypi hass[4833]: resp = self.send(prep, **send_kwargs)
Jun 15 01:43:55 raspberrypi hass[4833]: File "/srv/homeassistant/lib/python3.7/site-packages/requests/sessions.py", line 643, in send
Jun 15 01:43:55 raspberrypi hass[4833]: r = adapter.send(request, **kwargs)
Jun 15 01:43:55 raspberrypi hass[4833]: File "/srv/homeassistant/lib/python3.7/site-packages/requests/adapters.py", line 516, in send
Jun 15 01:43:55 raspberrypi hass[4833]: raise ConnectionError(e, request=request)
Jun 15 01:43:55 raspberrypi hass[4833]: requests.exceptions.ConnectionError: HTTPSConnectionPool(host='px1.tuyaus.com', port=443): Max retries exceeded with url: /homeassistant/skil
l (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0xa0314f70>: Failed to establish a new connection: [Errno 110] Connection timed out'))
It's making unusable my devices via HA.
estebanz01
on 15 Jun 2020
OK, so... I added this to /srv/homeassistant/lib/python3.7/site-packages/tuyaha/tuyaapi.py
import socket
import requests.packages.urllib3.util.connection as urllib3_cn
def allowed_gai_family():
family = socket.AF_INET
return family
urllib3_cn.allowed_gai_family = allowed_gai_family
after _LOGGER and now the error is gone and I can control my devices again without problems.
EDIT: and no more DNS request flooding.
estebanz01
on 15 Jun 2020
OK, so... I added this to
/srv/homeassistant/lib/python3.7/site-packages/tuyaha/tuyaapi.pyimport socket import requests.packages.urllib3.util.connection as urllib3_cn def allowed_gai_family(): family = socket.AF_INET return family urllib3_cn.allowed_gai_family = allowed_gai_familyafter
_LOGGERand now the error is gone and I can control my devices again without problems.EDIT: and no more DNS request flooding.
will this workaround work with HA installed in Docker? i see every 20-30 seconds lots of DNS queries to tuya flooding DNS. currently not able to manage my 20+ tuya switches with HA.
Thanks
donnaran
on 15 Jun 2020
OK, so... I added this to
/srv/homeassistant/lib/python3.7/site-packages/tuyaha/tuyaapi.pyimport socket import requests.packages.urllib3.util.connection as urllib3_cn def allowed_gai_family(): family = socket.AF_INET return family urllib3_cn.allowed_gai_family = allowed_gai_familyafter
_LOGGERand now the error is gone and I can control my devices again without problems.EDIT: and no more DNS request flooding.
Sorry, would you mind explaining exactly how your updates stop all the DNS requests to px1.tuyaeu.com ?
smithbill17
on 15 Jun 2020
I suggest to open an issue to tuyaha library because seems that you directly modify the library. Then we can come back here to open a PR for using new version of library with possible fix implemented. Also put reference to this Issue so that there is a link.
ollo69
on 15 Jun 2020
OK, so... I added this to
/srv/homeassistant/lib/python3.7/site-packages/tuyaha/tuyaapi.pyimport socket import requests.packages.urllib3.util.connection as urllib3_cn def allowed_gai_family(): family = socket.AF_INET return family urllib3_cn.allowed_gai_family = allowed_gai_familyafter
_LOGGERand now the error is gone and I can control my devices again without problems.
EDIT: and no more DNS request flooding.will this workaround work with HA installed in Docker? i see every 20-30 seconds lots of DNS queries to tuya flooding DNS. currently not able to manage my 20+ tuya switches with HA.
Thanks
I added this and it seems to work. Also without this fix this issue seems to only be affecting my multiple outlet switches, the singles seem unaffected at the moment.
Scope666
on 15 Jun 2020
@smithbill17 sure. What I'm doing is monkeypatching the urllib3 library that requests uses, so all connections, no matter what, are either downgraded or sticked with IPV4, so the library doesn't need to perform endless DNS queries trying to find the IPV6 IP of the tuya URL.
Since it's a monkeypatch I'm not sure it fits properly as a PR on the tuyaha library. I'm not sure if the problem is caused here on the HA core or on the library. I just did the change there just because it is where the raw request is made, but maybe a real fix may involve a change on HA core instead.
estebanz01
on 15 Jun 2020
@ollo69 do you think the issue is on tuyaha? 馃 see my previous comment about monkeypatching
estebanz01
on 15 Jun 2020
posted an issue on tuyaha https://github.com/PaulAnnekov/tuyaha/issues/36
estebanz01
on 16 Jun 2020
@ollo69 do you think the issue is on tuyaha? 馃 see my previous comment about monkeypatching
I don't have so much network experience to give you the right answer. Open the issue on TuyaHa for sure is correct bacause your mod refer to that, let's see codeowner opinion.
If you have a clear idea you can also try to open another issue here not related to the tuya component.
ollo69
on 16 Jun 2020
OK, so... I added this to
/srv/homeassistant/lib/python3.7/site-packages/tuyaha/tuyaapi.pyimport socket import requests.packages.urllib3.util.connection as urllib3_cn def allowed_gai_family(): family = socket.AF_INET return family urllib3_cn.allowed_gai_family = allowed_gai_familyafter
_LOGGERand now the error is gone and I can control my devices again without problems.EDIT: and no more DNS request flooding.
Any idea how those of us with Hass.io in Docker can apply this patch?
cannondale0815
on 17 Jun 2020
Any idea how those of us with Hass.io in Docker can apply this patch?
@estebanz01 opened a issue in the library repository.
Now we have to wait for library codeowner, then if a new release of the library will be released we can open a PR here to use the new release,
ollo69
on 17 Jun 2020
OK, so... I added this to
/srv/homeassistant/lib/python3.7/site-packages/tuyaha/tuyaapi.pyimport socket import requests.packages.urllib3.util.connection as urllib3_cn def allowed_gai_family(): family = socket.AF_INET return family urllib3_cn.allowed_gai_family = allowed_gai_familyafter
_LOGGERand now the error is gone and I can control my devices again without problems.
EDIT: and no more DNS request flooding.Any idea how those of us with Hass.io in Docker can apply this patch?
I did it directly in docker container (on my Pi the file is not in "/srv/homeassistant/lib/pyth...", it's actually in "/usr/local/lib/python3..."), but it doesn't work for me. The flooding didn't stop and also all Tuya entities are still not responding in HA. We'll just have to wait for the author to find the real cause (maybe changed API at Tuya servers???) and update the file.
AdmiralStipe
on 17 Jun 2020
Does anybody know when thsi will fixed ?
gomble
on 27 Jun 2020
Unfortunately, given that the original issue (#16109) was posted in Aug. 2018, I can't imagine it's going to be fixed any time soon (disappointing to say the least).
In the interim, you can use the 'localtuya' component - it requires a bit of work and messing around to get device IDs & local keys, but I can confirm it works for switches without any DNS flooding going on. I think this is the right link:
smithbill17
on 27 Jun 2020
... Or you can flash them with Tasmota via Tuya-Convert. Was a bit of a project but I'm glad I did it, local control is the way to go.
Scope666
on 27 Jun 2020
I have the same issue. Adguard DNS reports over 2 million DNS requests to tuya from Hass server in less than 30days:
Is there a fix for this issue?
Please help!!
tomx420
on 1 Jul 2020
First of all, I'm not an expert so What I say may be different... it's just a guess. (And English is an automatic translation)
The current implementation makes API calls for each device such as switches and lights to check the status. Previously, the API rarely returned an error, so that was fine. Unfortunately, since the beginning of last month, almost no requests have been accepted and an error is returned.
Therefore, we are trying to reduce the number of API calls by acquiring the status as a whole instead of checking each device and reducing the frequency.
It is exposed as a custom component so we can try it out immediately. It's easy because you just remove the current Tuya integration and switch to this custom component.
The custom component is available in the following repositories and can also be installed via HACS.
nao-pon
on 1 Jul 2020
My PR in tuyaha got merged now: https://github.com/PaulAnnekov/tuyaha/pull/38 so I expect a reduction of DNS queries once the new version hits HA.
estebanz01
on 10 Jul 2020
This is not immediate in HA. We have to wait for a new release of TuyaAPI, then I can PR in HA use of new library version. At this moment last version released of TuyaAPI is still 0.0.6. In any case this PR will not reduce the number of call to Tuya Cloud than is the main problem and causing "FrequentlyInvoke" error.
In any case this week-end I will add to my experimental repository "tuya_custom" your change with the use of session to see if provide additional benefit.
ollo69
on 10 Jul 2020
I have tested it custom and it works great. Expect a new Tuyaha API soon - was just merged today
djtimca
on 10 Jul 2020
I also released a new version of "tuya_custom" that contain the same fix (change was implemented by @nao-pon)
I see that @PaulAnnekov already PR to HA the new library version, so probably next release of HA will cointain the fix.
In any case I repeat that this fix is not resolving the "FrequentlyInvoke" error and so delay in update from Tuya Cloud to HA will not be resolved.
With Tuya Custom I will continue to search alternative solution for this.
ollo69
on 10 Jul 2020
There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.
Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 馃憤
This issue now has been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.
![stale[bot] picture](https://avatars3.githubusercontent.com/in/1724?v=4&s=40)
stale[bot]
on 12 Oct 2020
Related issues
aweb-01
路
3Comments
piitaya
路
3Comments
i-am-shodan
路
3Comments
coolriku
路
3Comments
sogeniusio
路
3Comments
Most helpful comment
I also released a new version of "tuya_custom" that contain the same fix (change was implemented by @nao-pon)
I see that @PaulAnnekov already PR to HA the new library version, so probably next release of HA will cointain the fix.
In any case I repeat that this fix is not resolving the "FrequentlyInvoke" error and so delay in update from Tuya Cloud to HA will not be resolved.
With Tuya Custom I will continue to search alternative solution for this.