Tasmota mqtt topic topology/hierarchy

[ThomDietrich]

Hey @arendst, this topic was on my mind for a longer time now.
Why did you choose the current mqtt topic structure and is it still the ideal design? MQTT topic structures are normally build following the real-world hierarchy of things, see best practices by HiveMQ or one alternative approach by homie.

Sonoff-Tasmota went a strangly different path by the top level elements cmnd and stat. I believe the following structure or any similar structure would give the end user certain benefits in regards to ease of use and in combination with mqtt wildcard subscriptions.

tasmota/mysonoff-device/stat/POWER   -> ON
tasmota/mysonoff-device/cmnd/power   <- ON

looking at the excellent homie convention, this could be further improved:

tasmota/mysonoff-device/power        -> ON
tasmota/mysonoff-device/power/set    <- ON

[arendst]

I suppose the wildcard subscriptions need to be handled by sonoff?

And tasmota/ looks redundant to me.

[ThomDietrich]

I meant wildcard subscriptions in the end user client. It's often useful to subscribe to +/sonoff-xyz/# or stat/+/LWT to observe devices on the network.

[arendst]

Just looked at homie suggestions. What about sensor status if there are 7 sensors. Do you really want 7 mqtt messages when sensor status is updated? I now prefer the JSON messages as it makes the air a lot cleaner.

[ThomDietrich]

"tasmota/" would be the common ground. It will be helpful if you want to subscribe to one topic of all tasmota devices. I would not at all call that redundant. However as I said, this is just one example.

homie does things differently and I just posted it as an example. The way Tasmota generates json formatted answers is fine with me.

This ticket is aimed at the current situation with cmnd and stat as the unlogical head of the hierarchy.

[arendst]

I do appreciate your input.

[ThomDietrich]

That's nice to hear. What's your opinion? I guess changing the mqtt topic structure would rattle a few users but maybe you want to consider it at some point!?

[arendst]

It's a major change so I need to schedule a new Sonoff-Tasmota-v2 ....

[ThomDietrich]

That big? How about Sonoff-Tasmota tag v4.5? :smiley:

[davidelang]

just a quick not that you are ignoring the tele/# top level element

I'm not sure if it's really easier to subscribe to

tasmota/+/tele/#

than it is to subscribe to

tele/#
(or optionally, with a slight edit to the config)
tasmota/tele/#

It depends on if youare looking at the bus from the point of view of devices or
functionality.

If you are looking at it in terms of devices, you are correct, it makes sense to
have the device name higher in the tree.

But if you are looking at it in terms of functionality (I want all stat messages
when devices turn on or off), it makes sense to have the message class higher.

with wildcards, either one can work, as long as one side or the other is
configurable.

It would be a nice enhancement to tasmota to allow the order to be configured.

[davidelang]

at least v5.x

but I think it would be better to have it be a config option.

On Mon, 20 Mar 2017, Thomas Dietrich wrote:

That big? How about Sonoff-Tasmota tag v4.5? :smiley:

[pauly7300]

This is an interesting topic. Yes, I agree it comes down to how you view your devices. The name of the device or the thing or things you have it configured to do. I suspect each person will have their own favorite 'mental model' so I would would vote for this being an option to allow the ordering if you actually consider implementing it. That should allow for no disruption for existing devices and provide the option for those that want the other.

[JonSilver]

I have to agree. Although there are many great reasons to commend the project and its features, the topic structure is awful, and the complete lack of flexibility over the choice of topic structure make the project cumbersome to fit into an existing infrastructure.

Honestly it'd be far better to implement topic settings in the way it's done in LetsControlIt/ESPEasy, where it can be done as static text or with optional plugin tokens.

[arendst]

I'm looking into changing topic options.
Currently I subscribe to cmnd/sonoff/# where

• cmnd = <TopicPrefix> and used to distinguish commands from status messages
• sonoff = <Topic> and used as device name

Both <TopicPrefix> and <Topic> are user configurable options both online via commands Prefix1 (10 characters) and Topic (32 characters) or statically in user_config.h

From the options mentioned here I sense an urge to swap both items and optionally add a static text in front of both items to distinguish tasmota devices from others.

• <Topic>/<TopicPrefix>/#
• tasmota/<Topic>/<TopicPrefix>/#

So it might be useful to have a global subscription string (as ESPEasy has) like

• tasmota/%Topic%/%TopicPrefix%/#

Where users can swap them around as they like. Problem will be input validation but I think I leave that to the user if things do not work as they had intended.

Feel free to shoot as I plan to implemented it shortly.

[davidelang]

Yes, devicetype/device/category/command seems to be what some mqtt software
assumes, so tasmota///command is what this translates to in
tasmota terms.

[JonSilver]

Whatever you do, don't make it a rigid structure of field1/field2/field3/whatever like it's been, because it makes no sense in a wider infrastructure where many different technologies share the same MQTT broker.

I tend to allow configuration of a topic root which can have its own structure, I.e. Have multiple parts delimited by /. Then there's a status subtopic under the topic root, and a command subtopic too, plus others that make sense. For example one of our sensor networks has an announcement topic where all units announce themselves upon startup. And if we want all units to download an OTA update, it's announced on a broadcast OTA topic, and units randomise their download slot.

tasmota/unit/cmd and tasmota/unit/telemetry etc definitely makes more sense than the current scheme. But if we can place the entire hierarchy under another hierarchy where all our actuation/control systems go, that would be even better.

[davidelang]

readingup on the mqtt spec, it only specifies 4 levels (a/b/c/d), it sound like
you want 5 (a/b/c/d/e)

if you have more than 4 levels, and you want different things for different
types of messages, how would you specify what schema to use for the different
types of things?

I may be misunderstanding you, but it sounds like you want to be able to do

foo/tasmota/sonoff1/status
bar/tasmota/sonoff1/tele
bax/tasmota/

for just one device, depending on the message.

[JonSilver]

No, just one root topic...

actuators/tasmota/floodlight/cmd
actuators/tasmota/floodlight/telemetry
Etc

[arendst]

I now have it functional with a default topic string of max length 100 characters which may contain plugin tokens for both current known Topic and Prefix. So it's something like this:

• tasmota/kitchen/%topic%/%prefix%/
• earth/europe/sanmarino/bedroom/%prefix%/%topic%/
• tasmote/device/%prefix%/
• %prefix%/%topic%/
• device

Again, biggest challenge is user input validation and avoiding mqtt topic loops.

[JonSilver]

That's great! I'll look forward to trying it!

Of course it depends just how suitable you want to make it for the non-techie user. After all, at present you have to be able to solder a header and use programming software and an MQTT broker to even get the firmware onto these devices and use them, so I'd hope users could get an appropriate topic structure sorted out. Maybe we could just put a big red box on the wiki saying be careful of MQTT looping as a result of a device subscribing to its own publications.

[arendst]

See wiki > MQTT features.

Give it a go and let me know what can be improved (and how too ;-))

[JonSilver]

Thanks Theo, I shall try it tomorrow morning (my wife might complain if I try it on her birthday!)
:)

[ThomDietrich]

Excellent work @arendst, I'll try soon but in theory everything sounds good. As you've already implemented this I'll close the issue, we can reopen if serious problems arise...

[davidelang]

check that you have the new config in use, see the troubleshooting page in the
wiki.

[TRIROG]

Figured it out.
I didn't know i had to reset the sonoff in order for the new config to be used.