Tasmota: Custom Tasmota MQTT discovery format to enable native HA integration

[emontnemery]

Have you looked for this feature in other issues and in the docs?
Yes

Is your feature request related to a problem? Please describe.
Home Assistant's MQTT discovery protocol includes a lot of redundant information which increases Tasmota's code size, and has some crucial limitations e.g. RGBxx lights flickering when turning on from HA.
Also, Home Assistant's MQTT discovery protocol is not easy to add new features or make breaking changes to.

Describe the solution you'd like
Acceptance of a custom MQTT discovery protocol for Tasmota. Work has started, but some feedback is needed.
An estimate is that 2-3kB code can be saved.

Short term, this will have to co-exist with Home Assistant's MQTT discovery format:
Maybe:

• When SetOption19 = 0, send Tasmota's custom MQTT discovery format
• When SetOption19 = 1, send Home Assistant's MQTT discovery format

Longer term, support for Home Assistant's MQTT discovery format can be removed from Tasmota or included as a non default compile-time option if needed by other SW, e.g. for OpenHab support.

Describe alternatives you've considered

1. Build a native HA integration but keep using HA's MQTT discovery protocol.
   Drawback: It will still not be possible to keep compliant to HA's MQTT discovery protocol and also add new features.
2. Don't use any discovery format at all, send MQTT messages to query the Tasmota device about its configuration.
   Drawback: Not all necessary information can (easily) be read by MQTT messages.

Additional context
Work on a Home Assistant native integration is started, with an MVP PR https://github.com/home-assistant/core/pull/39624 which supports:

• Discovery, update and removal of Tasmota devices
• Discovery, update and removal of relays (switch entities in HA)

The planned next steps are:

• Discovery, update and removal of lights
• Discovery, update and removal of switches
• Discovery, update and removal of buttons
• Discovery, update and removal of sensors
• Add support to manage the Tasmota from HA
• Add Tasmota console in HA
• Ideas??

Preliminary discovery message format example
Topic: tasmota/discovery/49A3BC
Payload:

{
  "ip":"192.168.15.10",                             // IP address
  "dn":"Living Room",                               // Device name
  "fn":["Ceiling Lamp", "Floor Lamp"],              // List of friendly names
  "hn":"tasmota_49A3BC-0956",                       // Hostname
  "id":"49A3BC",                                    // ChipID
  "md":"Sonoff Dual",                               // Module
  "of":"Offline",                                   // D_OFFLINE
  "on":"Online",                                    // D_ONLINE
  "st":["OFF","ON","TOGGLE","HOLD"],                // StateText[0..3]
  "bd":"8.4.0.2",                                   // Tasmota SW build version
  "t":"tasmota_49A3BC",                             // Topic
  "ft":"%prefix%/%topic%/",                         // Fulltopic
  "tp":["cmnd","stat","tele"],                      // [SUB_PREFIX, PUB_PREFIX, PUB_PREFIX2]
  "li":[0,0,0,0,0,0,0,0],                           // Lights, 0: disabled, 1: Enabled
  "rl":[0,0,0,0,0,0,0,0],                           // Relays, 0: disabled, 1: relay, 2.. future extension (fan, shutter?)
  "sw":[0,0,0,0,0,0,0,0],                           // Switches, 0: disabled: 1: enabled
  "bt":[0,0,0,0],                                   // Buttons, 0: disabled: 1: enabled
  "so":{"13":0,"17":1,"30":0,"37":1,"68":0,"73":1}, // SetOption needed by HA to map Tasmota devices to HA entities and triggers
  "lt_st":0,                                        // Light subtype
  "se":[0],                                         // Sensors, 0: disabled, 0..xx index in kHAssJsonSensorTypes (??)
  "ver":1                                           // Discovery protocol version, must be 1
}

Screenshots
Add Tasmota integration to HA:

Discovery message received by HA:

(Please, remember to close the issue when the problem has been addressed)

[blakadder]

My first concert is that a user will not be able to manually add entities under a device by sending the autodiscovery payload

See for what I mean:

https://blakadder.com/pir-in-tasmota/

As clunky as it is the current MQTT discovery has a wide range of possibilities

[arendst]

If this is the way HASS discovery can be made less tasmota version dependant it's fine with me.

As suggested pls leave the option to fallback to the current discovery process. In time we can decide to remove support for it and keep the new process only.

[digiblur]

Very cool stuff! I'm all for removing some steps and confusion with users. Not to mention saving code space as well! Well done!

[sfromis]

I came to wonder about a couple of small details with the new discovery.

Assuming that the new tasmota/discovery/<mac>/sensors message published to MQTT has the purpose of allowing the subscribing backend system to setup for what sensors and values can be expected, this seems to imply that the "model" sensor JSON payload should contain all relevant keys, but this may not be the case, even during entirely normal operation. Specifically, the ENERGY object does not contain the "Period" key; consumption within TelePeriod. I suppose that this makes sense, as it does not have a "start of TelePeriod" value to compare with. However, this would mean that a backend system might not be prepared for getting this value in a following message. Example:

tasmota/discovery/CC50E3******/sensors {"sn":{"Time":"2020-10-17T18:56:16.440+02:00","ENERGY":{"TotalStartTime":"2020-01-17T19:18:31","Total":261.244,"Yesterday":1.737,"Today":1.163,"Power":76.200,"ApparentPower":131.265,"ReactivePower":106.900,"Factor":0.58,"Voltage":231.100,"Current":0.568}},"ver":1}

Also, I'm used to all published MQTT messages to be logged in the console with MQT: tags, but this does not happen for the two messages for new discovery. In the nitpicky department, I'm a bit surprised that the MAC is uppercased in the topic, usually it is with lowercase a-f.

[emontnemery]

That's an interesting observation about the energy sensor, and you're correct that HA will need all keys in the initial message. What does the period key do?

The MQTT messages are hidden on purpose to not spam the log.

I don't think HA minds if MAC address is with lower or upper case letters 😁

[sfromis]

The Period field only appears for messages triggered by TelePeriod, and is energy usage in Wh for that interval, with decimals set by WattRes. It does not appear for extra reporting triggered by PowerDelta or Status 10, which seems to be similar to this initial reporting.

Example:
********/tele/SENSOR = {"Time":"2020-10-17T20:06:19.473+02:00","ENERGY":{"TotalStartTime":"2020-01-17T19:18:31","Total":261.428,"Yesterday":1.737,"Today":1.347,"Period":7.020,"Power":84.500,"ApparentPower":135.313,"ReactivePower":105.700,"Factor":0.62,"Voltage":231.700,"Current":0.584}}
For the standard TelePeriod of 5 minutes, the Period value of 7.020 would (if stable over one hour) be 84.240 Wh, pretty close to the Power value of 84.500 W (which is just the momental value at time of reporting).

[effelle]

Discovery /sensors topic grab everything is present on MqttShowSensor(); at boot time after mqtt init.
If something is not there means Tasmota has not initialized a particolar sensor at the time. That's the case of Period, becasue it needs two values to compare and compute.
There isn't much to do in this case @emontnemery , the only thing we could do is forcing the sensor creation directly in Python if a power meter is detected.

[sfromis]

I've also seen at least one case of a sensor not responding initially, but coming online a bit later, meaning that config based on the /sensors topic would be incomplete. I remember a case of a DS18B20 being "late", which can be said to be a issue with the sensor, but his could lead to confused users when it was not discovered backend.

[effelle]

I'll see if I can add a mild delay just to be sure.

[servim]

Forgive me if I'm not in the subject, I updated the old firmware to 8.5.1(tasmota) and found topics on the MQTT server that were incomprehensible to me:
tasmota/discovery/....

I do not use HA and autodiscover, how can I disable the publication of these topics?
Thanks!

[emontnemery]

They can be disabled by undefining USE_HOME_ASSISTANT if you build your own firmware.

[SteveDinn]

I've just upgraded a test tasmota device to v9.1.0, and added the Tasmota (beta) integration in Home Assistant (v0.117.4). I see the two retained tasmota/discovery messages when I watch things with mosquitto_client, but nothing is showing up under the integration in Home Assistant. Should there be something there? What are newly added entities named by default?

[emontnemery]

You're most likely hit by a bug which is fixed in Home Assistant 0.117.5 (released today).

[SteveDinn]

Arrrrgh...Got the notification email for that, but the docker container isn't available yet. I'll check out 0.117.5 when am able.

[servim]

They can be disabled by undefining USE_HOME_ASSISTANT if you build your own firmware.

What if I update with firmware files downloaded from here? I have no chance to disable the publication of these topics? Will there be such an opportunity in future firmware?

[1igeza]

I'm using newest dev version and HA 0.117.5 with new tasmota integration. In HA all temperature sensors are visible as 22,3 C, but it should be 22,3 °C.
Mqtt discovery message is:
{"sn":{"Time":"2020-11-08T10:16:13","BME280":{"Temperature":21.2,"Humidity":62.7,"DewPoint":13.8,"Pressure":969.6,"SeaPressure":1029.3},"PressureUnit":"hPa","TempUnit":"C"},"ver":1}

[emontnemery]

@1igeza thanks for reporting, that will be fixed in HA 0.118.

[f0ff886f]

Question, I'm trying to get this working with a Sonoff Basic R3, Tasmota 9.1.0, HA 0.177.5, and a mosquitto broker. I flashed with tasmota-lite.bin (as I understand the Basic R3 will not work with the full fat tasmota.bin

No matter what I do, nothing is discovered in HA. I can publish POWER TOGGLE to my switch in the HA MQTT configuration page and it toggles the switch just fine, so everything can communicate via MQTT successfully.

I have SetOption 19 = 0, but crucially no topic in tasmota/discovery exists at all (I see tele/tasmota_4CC23F for example though, so the switch is talking to MQTT).

Is there something required in the Tasmota FW? I haven't set anything out of the ordinary (MQTT settings other than host are all set to default, and I didn't configure anything else).

I don't have an option to set a discovery topic prefix when adding the Tasmota integration... it just added and has no configuration options available in HA. Sorry for posting here if HA would be the better spot, but I figure something is wrong with my config if my switch isn't publishing to the discovery topic at all.

[effelle]

tasmota-lite.bin has not discovery enabled.

[f0ff886f]

🤦 thank you. I was under the assumption that tasmota.bin wouldn't work (all instructions online talk about tasmota-lite.bin, but I could flash it from the Tasmota Web UI just fine. Auto-discovery is working perfectly now, thank you!

[sfromis]

The main point about using the smaller tasmota-lite as the first Tasmota binary is for OTA from other firmware. It is recommended to always upgrade to full Tasmota as the next step.

[ascillato2]

Closing this issue as this feature has been added.

Thanks for the hard work!