Core: Modbus RS485 unreliable communication

[SiliconAvatar]

Home Assistant release with the issue:
0.80.3

Last working Home Assistant release (if known):
Unknown (never?)

Operating environment (Hass.io/Docker/Windows/etc.):
Hassio running on Ubuntu 18.04.1 LTS on an Intel NUC

Component/platform:
Modbus component with Modbus Sensor

Description of problem:

I'm using a USB to Serial (RS485) converter to read the power information from two PZEM-016 energy meters on the same bus. They are assigned slave addresses of 1 and 2.

When I poll these using a traditional modbus program such as Modbus Poll I can read all registers of both devices very quickly and without error (faster than HA is set to poll them).

However when using Home Assistant to read these devices, I get up to 14 errors per second in the log.

The error that shows up is a variation of this error, but with the slave address and register is different for each error. It does not appear that any particular device or register is more prone to generating this error than any other, as they all seem to appear quite regularly.

No response from modbus slave 1, register 1
14:54 components/sensor/modbus.py (ERROR)
No response from modbus slave 1, register 0
14:54 components/sensor/modbus.py (ERROR)
No response from modbus slave 2, register 1
14:54 components/sensor/modbus.py (ERROR)
No response from modbus slave 1, register 0
14:54 components/sensor/modbus.py (ERROR)
No response from modbus slave 2, register 3
14:54 components/sensor/modbus.py (ERROR)
No response from modbus slave 1, register 7
14:54 components/sensor/modbus.py (ERROR)
No response from modbus slave 2, register 5
14:54 components/sensor/modbus.py (ERROR)
No response from modbus slave 2, register 7
14:54 components/sensor/modbus.py (ERROR)
No response from modbus slave 1, register 0
14:54 components/sensor/modbus.py (ERROR)
No response from modbus slave 2, register 5
14:54 components/sensor/modbus.py (ERROR)
No response from modbus slave 2, register 0
14:54 components/sensor/modbus.py (ERROR)
No response from modbus slave 1, register 3
14:54 components/sensor/modbus.py (ERROR)
No response from modbus slave 1, register 0
14:54 components/sensor/modbus.py (ERROR)
No response from modbus slave 2, register 8
14:54 components/sensor/modbus.py (ERROR)

The data does eventually get read, and I get valid numbers from them, however this is an extremely excessive amount of errors that is being generated.

Looking through the issue log, it seems like this issue was noticed previously, Issue #9451.
It also seems like a resolution was proposed and implemented, but never accepted into the main branch, PR #10967
.

I agree with the user of the above PR, that it's likely a timing issue, as in the professional world where modbus is used frequently, delay between queries, delay before tx, and delay before rx are very common setting to have to tweek to get values to read in reliably; not having these options in the HA implementation seems to be an oversight.

Could someone take a look at this issue, and possibly finish up the work started in PR #10967 so this can get accepted into the main branch?

Thanks.

Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):

modbus:
  type: serial
  method: rtu
  port: /dev/ttyUSB0
  timeout: 2
  baudrate: 9600
  stopbits: 1
  bytesize: 8
  parity: N

sensor:
- platform: modbus
  scan_interval: 15
  registers:
    - name: HousePower_PhaseA_Voltage
      unit_of_measurement: V
      slave: 1
      register: 0
      register_type: input
      scale: 0.1
      offset: 0
      precision: 1
    - name: HousePower_PhaseA_Current
      unit_of_measurement: A
      slave: 1
      register: 1
      count: 2
      reverse_order: true
      register_type: input
      scale: 0.001
      offset: 0
      precision: 1
    - name: HousePower_PhaseA_Power
      unit_of_measurement: W
      slave: 1
      register: 3
      count: 2
      reverse_order: true
      register_type: input
      scale: 0.1
      offset: 0
      precision: 1
    - name: HousePower_PhaseA_Wh
      unit_of_measurement: Wh
      slave: 1
      register: 5
      count: 2
      reverse_order: true
      register_type: input
      scale: 1
      offset: 0
      precision: 1
    - name: HousePower_PhaseA_Frequency
      unit_of_measurement: Hz
      slave: 1
      register: 7
      register_type: input
      scale: 0.1
      offset: 0
      precision: 1
    - name: HousePower_PhaseA_PowerFactor
      unit_of_measurement: Pf
      slave: 1
      register: 8
      register_type: input
      scale: 0.01
      offset: 0
      precision: 1
    - name: HousePower_PhaseB_Voltage
      unit_of_measurement: V
      slave: 2
      register: 0
      register_type: input
      scale: 0.1
      offset: 0
      precision: 1
    - name: HousePower_PhaseB_Current
      unit_of_measurement: A
      slave: 2
      register: 1
      count: 2
      reverse_order: true
      register_type: input
      scale: 0.001
      offset: 0
      precision: 1
    - name: HousePower_PhaseB_Power
      unit_of_measurement: W
      slave: 2
      register: 3
      count: 2
      reverse_order: true
      register_type: input
      scale: 0.1
      offset: 0
      precision: 1
    - name: HousePower_PhaseB_Wh
      unit_of_measurement: Wh
      slave: 2
      register: 5
      count: 2
      reverse_order: true
      register_type: input
      scale: 1
      offset: 0
      precision: 1
    - name: HousePower_PhaseB_Frequency
      unit_of_measurement: Hz
      slave: 2
      register: 7
      register_type: input
      scale: 0.1
      offset: 0
      precision: 1
    - name: HousePower_PhaseB_PowerFactor
      unit_of_measurement: Pf
      slave: 2
      register: 8
      register_type: input
      scale: 0.01
      offset: 0
      precision: 1

Traceback (if applicable):

Additional information:

[polcape]

Hello @SiliconAvatar,

I notice the same problem, but with Modbus TCP devices.

[mindakas]

Hello,
I have problems with ModBus stability too (using TCP). It seems there were number of fixes done by the author of pymodbus (https://github.com/riptideio/pymodbus) in recent versions. I am a rookie in this area, but would it be possible to update component in Home Assistant v1.3.1 -> v1.5.2?

Symptoms for a reference - intermittently ModBus TCP updates sensor data with value from wrong Modbus register (e.g. Watt -> Voltage L1, KWh -> Voltage L2 - apparently some issue to read incoming data, but unfortunately I don't know how to debug more), and then crashes:

2018-11-25 20:27:02 ERROR (Thread-19) [homeassistant.components.sensor.modbus] No response from modbus slave 1, register 50
2018-11-25 20:27:57 ERROR (MainThread) [homeassistant.helpers.entity] Update for sensor.kwh fails Traceback (most recent call last): 
File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/helpers/entity.py", line 221, in async_update_ha_state await self.async_device_update() 
File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/helpers/entity.py", line 349, in async_device_update await self.hass.async_add_executor_job(self.update) 
File "/usr/lib/python3.5/asyncio/futures.py", line 380, in __iter__ yield self # This tells Task to wait for completion. 
File "/usr/lib/python3.5/asyncio/tasks.py", line 304, in _wakeup future.result() File "/usr/lib/python3.5/asyncio/futures.py", line 293, in result raise self._exception
File "/usr/lib/python3.5/concurrent/futures/thread.py", line 55, in run result = self.fn(*self.args, **self.kwargs) 
File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/components/sensor/modbus.py", line 160, in update self._count) 
File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/components/modbus.py", line 203, in read_holding_registers **kwargs) 
File "/srv/homeassistant/lib/python3.5/site-packages/pymodbus/client/common.py", line 109, in read_holding_registers return self.execute(request) 
File "/srv/homeassistant/lib/python3.5/site-packages/pymodbus/client/sync.py", line 85, in execute return self.transaction.execute(request) 
File "/srv/homeassistant/lib/python3.5/site-packages/pymodbus/transaction.py", line 140, in execute self.client.framer.processIncomingPacket(result, self.addTransaction) 
File "/srv/homeassistant/lib/python3.5/site-packages/pymodbus/transaction.py", line 420, in processIncomingPacket self.addToFrame(data) 
File "/srv/homeassistant/lib/python3.5/site-packages/pymodbus/transaction.py", line 380, in addToFrame self.__buffer += message TypeError: Can't convert 'bytes' object to str implicitly

configuration.yaml

- platform: modbus
    registers:
    - name: Voltage L1
      unit_of_measurement: V
      slave: 1
      register: 0
      count: 2
      reverse_order: 1
      scale: 0.1
    - name: Voltage L2
      unit_of_measurement: V
      slave: 1
      register: 2
      count: 2
      reverse_order: 1
      scale: 0.1
    - name: Voltage L3
      unit_of_measurement: V
      slave: 1
      register: 4
      count: 2
      reverse_order: 1
      scale: 0.1
    - name: Watt
      unit_of_measurement: W
      slave: 1
      register: 40
      count: 2
      reverse_order: 1
      scale: 0.1
    - name: KWh
      unit_of_measurement: KWh
      slave: 1
      register: 52
      count: 2
      reverse_order: 1
      scale: 0.1
    - name: Phase
      slave: 1
      register: 50
      scale: 0.1 

[mindakas]

Hi,
I've bumped pymodbus to version 1.5.2 on my local hassbian installation.

(homeassistant) homeassistant@hassbian:/srv/homeassistant $ pip3 show pymodbus
Name: pymodbus
Version: 1.5.2
Summary: A fully featured modbus protocol stack in python
Home-page: https://github.com/riptideio/pymodbus/
Author: Galen Collins
Author-email: [email protected]
License: BSD
Location: /srv/homeassistant/lib/python3.5/site-packages
Requires: pyserial
Required-by:

And made respective changes in modbus.py

homeassistant@hassbian:/srv/homeassistant/lib/python3.5/site-packages/homeassistant/components $ more modbus.py | grep pymodbus
REQUIREMENTS = ['pymodbus==1.5.2']

I still get intermittent ModBus data reading errors, but modbus.py doesn't crash anymore.

2018-12-01 14:10:43 ERROR (Thread-7) [homeassistant.components.sensor.modbus] No response from modbus slave 1, register 4
2018-12-01 14:10:46 ERROR (Thread-22) [homeassistant.components.sensor.modbus] No response from modbus slave 1, register 2

Sorry, I lack skills to propose version change on github, but the ones having same issue can give it a try.

[SiliconAvatar]

I bought a Modbus RTU -> Modbus TCP/IP converter (USR-W610) because I was fed up with this constant flooding of my logs.
Everything is working fine now, reading the same data as before, with the exact same wiring as before.

The Modbus RTU component in Home-Assistant is definitely broken in some way.
If someone gets around to fixing it I'll be glad to go back to the old converter and test it out, but for now it's not really usable if you want to have useful logs.

[poratnir1]

Hey
so i am bit confused...

both issues on git were closed and marked as solved

are you saying that RTU is still not working properly ?
"The Modbus RTU component in Home-Assistant is definitely broken in some way."
???

[SiliconAvatar]

are you saying that RTU is still not working properly ?

Correct, last I used the Modbus RTU component with more than one slave I was still getting flooded with the errors described in the original issue description.

I've moved to using a Modbus RTU->TCP/IP converter so I can use the Modbus TCP component in home-assistant, which does not flood the log with errors.

I don't see any changes to the component in github since I switched, so I have no reason to believe it was resolved.

[poratnir1]

Hey,
If I don't get errors but get only warnings..
Should it be ok?
Can I connect regular rs485 two wire to tcp?

[SiliconAvatar]

In my own case, I was getting errors and not warning, but the messages made it through often enough that I still got usable data, it just made the log unusable due to being flooded with 14+ errors per second.

If you are only getting warning, but still getting data then it should be fine unless you want to have usable logs.

As for your last question, I'm not sure what you mean. For 2-wire RS485 Modbus you must use the RTU component. If you are using modbus TCP/IP then it's over your LAN network so use the TCP component.

[poratnir1]

I am using RTU component , however there might be a converter that will translate it into TCP then i can use TCP for RTU communication.

which log are we talking about?

[SiliconAvatar]

however there might be a converter that will translate it into TCP then i can use TCP for RTU communication.

Yeah, that's what I'm using now. I'm using the USR-W610 Modbus RTU->TCP/IP converter.

which log are we talking about?

The log shown in the dev-info panel.

[poratnir1]

20 M max...

On Mon, Dec 31, 2018 at 3:27 AM SiliconAvatar [e-mail]
wrote:

however there might be a converter that will translate it into TCP then i
can use TCP for RTU communication.

Yeah, that's what I'm using now. I'm using the USR-W610 Modbus RTU->TCP/IP
converter.

which log are we talking about?

The log shown in the dev-info panel.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/home-assistant/home-assistant/issues/18128#issuecomment-450599552,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AhCMeFo8NGKSI6UduKHsLNsBqA6Kctfhks5u-WgLgaJpZM4YMNwS
.

[jara1]

Same issue here with 0.85.0 on LXC on QNAP with FTDI based RS485 adapter. I checked termination, wiring and all the HW and it is OK. Working perfectly with other tools.

I had two modbus devices and it was OK. When I addded third (power meter) it started to fill the log with ModBus data reading errors. Partially helped to set different scan_interval in HA. But errors are still there (with lower frequency).

I'm going to do more investigating with oscilloscope.

[jara1]

In the meantime I tested PR #10967 with delay_between_queries: 30 in my configuration.yaml and after hour no error. Looks like this is the solution...

[ziotibia81]

Question for @balloob , @fabaff and @benvm :

with latest chages to modbus hub to support multiple hubs, does each hub run in a separate thread different than the main thread?

If yes I could rake up, and adapt, my old PR #10967 that solve this issue.
In a RS485 (half duplex) bus there is need of "some silence" between queries to give the devices time to switch from TX to RX and electrically release the bus.

[danilKh]

In the meantime I tested PR #10967 with delay_between_queries: 30 in my configuration.yaml and after hour no error. Looks like this is the solution...

I tried it and has a massage:
Invalid config for [modbus]: [delay_between_queries] is an invalid option for [modbus].

[stale[bot]]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.
Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍
This issue now has been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

[SiliconAvatar]

This issue has not been resolved yet.

[allmoney]

I have same problem. A lot of errors:

No response from hub default, slave 2, register 7
18:00 components/modbus/sensor.py (ERROR) - message first occured at 3 октября 2019 г., 21:57 and shows up 866 times
No response from hub default, slave 1, register 6
18:00 components/modbus/sensor.py (ERROR) - message first occured at 3 октября 2019 г., 21:56 and shows up 829 times
Cleanup recv buffer before send: 0xfc
18:00 components/modbus/__init__.py (WARNING) - message first occured at 3 октября 2019 г., 21:59 and shows up 225 times
No response from hub default, slave 5, register 10
18:00 components/modbus/sensor.py (ERROR) - message first occured at 3 октября 2019 г., 21:56 and shows up 993 times
No response from hub default, slave 1, register 10
18:00 components/modbus/sensor.py (ERROR) - message first occured at 3 октября 2019 г., 21:57 and shows up 846 times
Cleanup recv buffer before send: 0x0
18:00 components/modbus/__init__.py (WARNING) - message first occured at 3 октября 2019 г., 22:12 and shows up 106 times

[remyderuysscher]

In the meantime I tested PR #10967 with delay_between_queries: 30 in my configuration.yaml and after hour no error. Looks like this is the solution...

That's not a valid configuration option.

[amonhk]

@ziotibia81 did you succeed in inserting option delay_between_queries: 30 in new versions of homeassistant?

[remyderuysscher]

@amonhk No this doesn't work. Tried it on 0.100.2

[ziotibia81]

@amonhk PR #10967 was discarded by maintainers. Apply this solution involves to almost rewite entire modbus hub to run it in in a different thread. I spent a lot of time on it but I can't undertand how do that. If someone will help me I will be very happy to fix modbus hub.

I can suggest you a workaround if you are using hass.io:

1. Add a MQTT broker
2. Add Node-red
3. Convert all your modbus entities in MQTT entities
4. Create a flow to interact with modbus device (https://flows.nodered.org/node/node-red-contrib-modbus - flex-write and flex-getter components)

[amonhk]

you mean to pass control of the RS485 bus to Node-red?
in this logical configuration:
modbus/rs485 -> Node-red -> MQTT -> hass.io

I like it, did you try this in "production"?

[ziotibia81]

@amonhk your deduction is correct.
I've not tried this configuration in a production environment yet but it should work.

[jara1]

In newer versions it is needed to create custom component and add delays to it. What I am using for quite long time is attached.

Delay function is located in __init__.py time.sleep(0.04)
modbus.zip

[remyderuysscher]

@jara1

In newer versions it is needed to create custom component and add delays to it. What I am using for quite long time is attached.

Delay function is located in __init__.py time.sleep(0.04)
modbus.zip

How can I get HA use the custom component instead of the build-in modbus version? I've installed the custom component (ZIP you provided) but delay commands are not recognised?

```
modbus:
name: hub1
type: serial
method: rtu
port: /dev/ttyUSB0
baudrate: 9600
stopbits: 1
bytesize: 8
parity: N
timeout: 4
delay_between_queries: 100
delay_before_tx: 10
delay_before_rx: 10
rts_level_for_tx: 1
rts_level_for_rx: 0

```

[amonhk]

@jara1 thank you, I tried your custom component and the errors have decreased

2019-10-20 11:27:26 ERROR (SyncWorker_4) [custom_components.modbus.sensor] No response from hub plcbus, slave 1, register 1212

2019-10-20 11:29:44 ERROR (SyncWorker_9) [custom_components.modbus.switch] No response from hub plcbus, slave 1, coil 6580

@remyderuysscher custom component does not handle additional commands but inserts a delay for each read / write call. See the __init__.py

I will also try the Node-red solution, maybe I'll have more control of the rs485 / modbus

[stale[bot]]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.
Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍
This issue now has been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

[ziotibia81]

This issue still exists

[tiagofreire-pt]

Suffering form the same issue, on 0.106.6.

Is there any assigned for this integration, please?

[nilvanis]

Same issue on Hassos 3.12, HA version 0.107.5.
I have two MODBUS devices connected and apparently only one is affected by that problem, even if it's the only one polled.
Problematic device is
https://www.fif.com.pl/en/usage-electric-power-meters/339-energy-meter-le-03m.html
It looks like only one register at a time gets "No response from hub" error.

[tiagofreire-pt]

@adamchengtkc , could give a hand on this, please?

[tiagofreire-pt]

May be solved by https://github.com/home-assistant/core/pull/32557?

[janiversen]

May be solved by https://github.com/home-assistant/core/pull/32557?

Let me know if it is not solved.

I also have a followup PR (not public yet) that adds a delay between each read/wrie.

[nilvanis]

May be solved by #32557?

Let me know if it is not solved.

I also have a followup PR (not public yet) that adds a delay between each read/wrie.

What I frogot to mention is that I use a serial connection over a USB converter.
Any chance for improvement in my case?

modbus:
  name: hub1
  type: serial
  method: rtu
  port: /dev/ttyUSB0
  baudrate: 9600
  stopbits: 1
  bytesize: 8
  parity: N  

[janiversen]

That just happens to be what I have on my mac, I also have a TTL-modbus converter on a raspberry so yes chances are good.

new parameter is likely to be:
message_delay =
and will be applied after each read/wrie.

But first I need to my PR rounded of and merged, then I can submit the second ine.