Core: Component mqtt binary_sensor log warning “No matching payload found for entity”

[wmrato]

Home Assistant release with the issue:
0.89.2

Last working Home Assistant release (if known):
unsure maybe (0.89.0+);
Whichever made custom-components require all component files.

Operating environment (Hass.io/Docker/Windows/etc.):
Hass.io

Component/platform:
https://www.home-assistant.io/components/binary_sensor.mqtt/

Description of problem:
When using multiple Sonoff_RF components, Error log outputs multiple
"[homeassistant.components.mqtt.binary_sensor] No matching payload found for entity: "

• Used to add Custom_component for mqtt/binary_sensor.py which would disable output.
  -- Now since 8.9.0 having to add all mqtt files via Configurator IS UN-WORKABLE.
  -- also monitoring release updates to all mqtt component files against custom-components IS UN-WORKABLE.

• Hass.io runs slow when issue outputs to log.

• Hass.io takes 15-35 minutes to reboot.
• MariadB is locking up due to too many of these errors recorded.
  -- Snapshot takes 90 mins, as MariaDB takes ages.
  -- this is eating up my SD Card's life expectancy.
  -- Don't like losing my house data.

Please revert pull request (https://github.com/home-assistant/home-assistant/pull/11836)
or update code from components/mqtt/binary_sensor.py #Line: 147
To comment out LOGGER warning
Example of change:

        @callback
        def state_message_received(_topic, payload, _qos):
            """Handle a new received MQTT state message."""
            value_template = self._config.get(CONF_VALUE_TEMPLATE)
            if value_template is not None:
                payload = value_template.async_render_with_possible_json_value(
                    payload, variables={'entity_id': self.entity_id})
            if payload == self._config.get(CONF_PAYLOAD_ON):
                self._state = True
            elif payload == self._config.get(CONF_PAYLOAD_OFF):
                self._state = False
            else:  # Payload is not for this entity
#                _LOGGER.warning('No matching payload found'
#                                ' for entity: %s with state_topic: %s',
#                                self._config.get(CONF_NAME),
#                                self._config.get(CONF_STATE_TOPIC))
                return

Example Error log (1 entry below, but logs this 19 times):

2019-03-19 10:23:09 WARNING (MainThread) [homeassistant.components.mqtt.binary_sensor] No matching payload found for entity: pir a hall with state_topic: tele/sonoff-RF/RESULT
2019-03-19 10:23:09 WARNING (MainThread) [homeassistant.components.mqtt.binary_sensor] No matching payload found for entity: pir b bathroom with state_topic: tele/sonoff-RF/RESULT
2019-03-19 10:23:09 WARNING (MainThread) [homeassistant.components.mqtt.binary_sensor] No matching payload found for entity: kitchen smoke with state_topic: tele/sonoff-RF/RESULT
2019-03-19 10:23:09 WARNING (MainThread) [homeassistant.components.mqtt.binary_sensor] No matching payload found for entity: livingroom window r with state_topic: tele/sonoff-RF/RESULT
2019-03-19 10:23:09 WARNING (MainThread) [homeassistant.components.mqtt.binary_sensor] No matching payload found for entity: kitchen window with state_topic: tele/sonoff-RF/RESULT
2019-03-19 10:23:09 WARNING (MainThread) [homeassistant.components.mqtt.binary_sensor] No matching payload found for entity: bedroom window with state_topic: tele/sonoff-RF/RESULT
2019-03-19 10:23:09 WARNING (MainThread) [homeassistant.components.mqtt.binary_sensor] No matching payload found for entity: livingroom window l with state_topic: tele/sonoff-RF/RESULT
2019-03-19 10:23:09 WARNING (MainThread) [homeassistant.components.mqtt.binary_sensor] No matching payload found for entity: front door with state_topic: tele/sonoff-RF/RESULT
2019-03-19 10:23:09 WARNING (MainThread) [homeassistant.components.mqtt.binary_sensor] No matching payload found for entity: kitchen fridge door with state_topic: tele/sonoff-RF/RESULT
2019-03-19 10:23:09 WARNING (MainThread) [homeassistant.components.mqtt.binary_sensor] No matching payload found for entity: remote1 ab with state_topic: tele/sonoff-RF/RESULT
2019-03-19 10:23:09 WARNING (MainThread) [homeassistant.components.mqtt.binary_sensor] No matching payload found for entity: remote1 c with state_topic: tele/sonoff-RF/RESULT
2019-03-19 10:23:09 WARNING (MainThread) [homeassistant.components.mqtt.binary_sensor] No matching payload found for entity: remote1 cd with state_topic: tele/sonoff-RF/RESULT
2019-03-19 10:23:09 WARNING (MainThread) [homeassistant.components.mqtt.binary_sensor] No matching payload found for entity: remote1 d with state_topic: tele/sonoff-RF/RESULT
2019-03-19 10:23:09 WARNING (MainThread) [homeassistant.components.mqtt.binary_sensor] No matching payload found for entity: remote1 a with state_topic: tele/sonoff-RF/RESULT
2019-03-19 10:23:09 WARNING (MainThread) [homeassistant.components.mqtt.binary_sensor] No matching payload found for entity: remote1 b with state_topic: tele/sonoff-RF/RESULT
2019-03-19 10:23:09 WARNING (MainThread) [homeassistant.components.mqtt.binary_sensor] No matching payload found for entity: pir e bedroom with state_topic: tele/sonoff-RF/RESULT
2019-03-19 10:23:09 WARNING (MainThread) [homeassistant.components.mqtt.binary_sensor] No matching payload found for entity: pir d livingroom with state_topic: tele/sonoff-RF/RESULT
2019-03-19 10:23:09 WARNING (MainThread) [homeassistant.components.mqtt.binary_sensor] No matching payload found for entity: pir c kitchen with state_topic: tele/sonoff-RF/RESULT

Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):

binary_sensor:
# Door
  - platform: mqtt
    name: 'front door'
    payload_on: 'SECRET_RF_CODE_ON'
    payload_off: 'SECRET_RF_CODE_OFF'
    device_class: door
    qos: 1
    state_topic: 'tele/sonoff-RF/RESULT'
    value_template: "{{ value_json.RfReceived.Data }}"
  - platform: mqtt
    name: kitchen fridge door
    payload_on: 'SECRET_RF_CODE_ON'
    payload_off: 'SECRET_RF_CODE_OFF'
    device_class: door
    qos: 1
    state_topic: 'tele/sonoff-RF/RESULT'
    value_template: "{{ value_json.RfReceived.Data }}"

# Window
  - platform: mqtt
    name: bedroom window
    payload_on: 'SECRET_RF_CODE_ON'
    payload_off: 'SECRET_RF_CODE_OFF'
    device_class: window
    qos: 1
    state_topic: 'tele/sonoff-RF/RESULT'
    value_template: "{{ value_json.RfReceived.Data }}"
  - platform: mqtt
    name: livingroom window l
    payload_on: 'SECRET_RF_CODE_ON'
    payload_off: 'SECRET_RF_CODE_OFF'
    device_class: window
    qos: 1
    state_topic: 'tele/sonoff-RF/RESULT'
    value_template: "{{ value_json.RfReceived.Data }}"
  - platform: mqtt
    name: livingroom window r
    payload_on: 'SECRET_RF_CODE_ON'
    payload_off: 'SECRET_RF_CODE_OFF'
    device_class: window
    qos: 1
    state_topic: 'tele/sonoff-RF/RESULT'
    value_template: "{{ value_json.RfReceived.Data }}"
  - platform: mqtt
    name: kitchen window
    payload_on: 'SECRET_RF_CODE_ON'
    payload_off: 'SECRET_RF_CODE_OFF'
    device_class: window
    qos: 1
    state_topic: 'tele/sonoff-RF/RESULT'
    value_template: "{{ value_json.RfReceived.Data }}"

Traceback (if applicable):

Additional information:
This is related to this change
https://github.com/home-assistant/home-assistant/pull/11836

plus referenced in
https://community.home-assistant.io/t/mqtt-py-log-warning-no-matching-payload-found-for-entity/42995/5

I really need this fixed, please let me know what I can do to fix this,
am I required to push a commit to fix / add configuration option to allow for Sonoff_RF or other devices acting on different codes.

[emontnemery]

Quick fix: change log level of homeassistant.components.mqtt.binary_sensor:

logger:
  default: info
  logs:
    homeassistant.components.mqtt.binary_sensor: critical

[emontnemery]

The reason why this is logged as warnings is to aid debugging, but maybe we should log a warning only once and then downgrade to debug severity.

[wmrato]

Wow thank you @emontnemery, your solution worked a charm.
I agree, as its a debug level event & worth downgrading to a debug entry
Should I close this issue or wait for debug entry change ?

[tdejneka]

The reason why this is logged as warnings is to aid debugging

Precisely.

The problem is multiple sensors using the same MQTT topic. Sometimes the payload is intended for Sensor A, sometimes it's for Sensor B. Nevertheless they _both_ receive the same payload and, of course, one fails to translate it into a valid state (thereby producing the warning message about 'no matching payload').

The solution is not to suppress legitimate warning messages but to improve the entity's value_template so it _always returns a valid state_. It's really that simple.

In this post I provided two examples of how to create a value_template that converts the received payload into a valid state.

[KennethLavrsen]

I do not agree that we should all turn something that is simple YAML into a very complicated use of value_templates and code I would call a hack. The issue is the logging of something that is not an error and that does not really help anyone. The long thread on https://community.home-assistant.io/t/mqtt-py-log-warning-no-matching-payload-found-for-entity/42995/82 is a very convincing argument why this logging should be reverted. I have personally altered the log level of the mqtt component but it does not remove the logging from the /dev-info page and this is where is really annoys me at the many many any loggings there drown the real issues that require attention.

There is so much attention right now on making HA easier for beginners. Requiring that a binary sensor using MQTT with popular projects like Tasmota is setup using a complex Jinja language hack is not beginner friendly. Neither is the 1000s of errors you get in a normal house with a few RF motion sensors that are triggered all the time as the family walks around the house.

[tdejneka]

I would call a hack

Here's what I would call a hack: a _single_ MQTT topic used to represent the states of _multiple_ sensors.

This technique flies in the face of MQTT's best practices. MQTT topics cost nothing and Home Assistant adheres to the well-established practice of supporting one or more topics _per sensor_ (not the odd practice of multiple sensors per topic).
https://www.hivemq.com/blog/mqtt-essentials-part-5-mqtt-topics-best-practices/

The issue is the logging of something that is not an error and that does not really help anyone.

Except it is an error and I want errors to be identified. If I agree to ship you black or white paint and you receive a drum of red paint, that's an error. Similarly, receiving a value that matches neither payload_on or payload_off is an error and ought to be reported.

For the case of devices that multiplex many sensors in one MQTT topic, the solution is to use a template to extract the correct values. That's standard procedure for JSON payloads. The template needed to extract values from the multiplexed topic isn't particularly complex.

An alternative solution for handling a multiplexed topic, especially when many sensors are involved, is to use a 'demultiplexer' automation to separate the multiplexed values into unique topics, one per sensor. The result is that each sensor's configuration requires no template.

I explained both techniques, in detail, in the following post:
Sonoff RF Bridge. Strategies for receiving data.

[KennethLavrsen]

Welcome to the real world my friend.

The real world consists of Tasmota which is used by 10000 of HA users and 1000s use an RF bridge which converts 433 MHz messages to MQTT. And it does that in a single topic. And then there is OpenMQTTGateway also used by many and it can be made to either send in a single topic or it can append the data payload to the topic path. That is great for Motion sensors with one code. But it cannot be used for a single binary_sensor in HA because it can only listen to one topic. Then you have to use a hack of input_booleans and template sensors to turn a single sensor on and off.

You would have to implement these RF gateways so they decode messages in pairs of two codes per mqtt topic. Not easy. Tasmota has a very geeky command line rule feature that can do it but there is only room for few rules. The right place is to do it in HA in the definition of a binary sensor with on and off payloads. And it is implemented and it works great with the number of sensors that normal people have in a normal home. The issue is just the error messages. If they were debug only and only visible when you change logging to debug then it would be OK. But the issue is that it floods the report in the web UI

I use the Jinja hack to avoid the error mssages. Yes it works for you and for me, two geeks. But it annoys me on behalf of the new users that are not programmers that things have to be so geeky because someone insists on giving error messages that few can use for anything

Alternatively the error could be displayed if a given mqtt topic has a payload that does not match any sensors or automations. But that would be harder and not just removing 3 lines of code

[tdejneka]

I use the Jinja hack

Except it's not a _hack_, its merely a template to extract values as is _commonly used for many other purposes in Home Assistant_.

The number of users is immaterial for what is a fundamentally poor implementation of MQTT topic structure. Fortunately, it can be handled by Home Assistant but not at the expense of becoming lax about handling errors. Receiving any value other than the two defined in the sensor's configuration is indeed an _error_ and I support it being reported as such. To avoid the error, use a template as is the custom for many other situations. Just par for the Home Assistant course and nothing 'geeky' about it.

[stale[bot]]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.
Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍
This issue now has been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

[DeviousPenguin]

Is it possible to re-open this issue? I've just started to get the same

‘No matching payload found for entity’

error on all devices that I have upgraded to Tasmota v6.7 from v6.6.

Or would it be better to open a new issue instead of re-opening this one?

[effelle]

@emontnemery Erik,
in my last update of Tasmota discovery .ino I've changed buttons and switches creation, following a new separated topic from relay to give a more granular control:

{
  "name": "Sonoff Switch1",
  "stat_t": "~stat/RESULT",
  "value_template": "{{value_json.SWITCH1}}",
  "pl_on": "ON",
  "avty_t": "~tele/LWT",
  "pl_avail": "Online",
  "pl_not_avail": "Offline",
  "uniq_id": "471078_SW_1",
  "device": {
    "identifiers": [
      "471078"
    ],
    "connections": [
      [
        "mac",
        "CC:50:E3:47:10:78"
      ]
    ]
  },
  "~": "sonoff/",
  "frc_upd": true,
  "pl_off": "OFF"
}

Some users are complains about the warning.
What should I change on json creation to match HA YAML?
Thanks.