Zwavejs2mqtt: [bug] Node rename is not always respected in MQTT

[lazyoldbear]

Official container zwavejs/zwavejs2mqtt:1.0.0-alpha.2.
Mosquitto 2.0.4.
Gateway is set for ValueID topics with enforced using of node names.
Devices are created without names. Now, when I add a name for them, it is saved in ZWave, all good.
For some devices (smart plugs) the topics in MQTT get updated, and once I reload HomeAssistant (after killing the devices there), I get pretty names for devices and entities.
Unfortunately, for Fibaro buttons (FGPB101) they stay as nodeXX, no matter what. Tried removing and healing in several ways. The name in ZWave is updated, MQTT topics are always numbered.

Please advise me what information to provide to help understand the trouble.

[larstobi]

I get this as well.

[AlCalzone]

Is there a difference in support for the node naming and location cc between the devices?

[robertsLando]

@AlCalzone No worries this is not something related to zwavejs. I never use node naming an location cc as I use my custom db for them as them were badly managed on ozw

[robertsLando]

@lazyoldbear Could you try to use dev tag? @larstobi Are you using latest version from master? Could someone show me an export of the node with this problem?

[larstobi]

node_93.json.zip

[larstobi]

I'm currently running:
zwavejs/zwavejs2mqtt:dev (9ad34d1bf555) 4 days ago

[larstobi]

You'll see:

  "_name": "temperatur_hovedsoverom"

but still:

      "discoveryTopic": "binary_sensor/nodeID_93/battery_islow/config",

[larstobi]

My entity name template is just "%n"

[robertsLando]

@larstobi Is this only happening on discovery topic? Or also in mqtt publish?

[larstobi]

[larstobi]

[robertsLando]

So to answer my question the problem seems to happen only on discovery topic. The mqtt side is ok

[lazyoldbear]

Worked on dev (7e519cef465bf7eee4b3db54b8a6c85c2956f212c682823bb3732a1d07f1bd4e).
Had to alter the name template to %n_%o, so that it does not put a dash (I feel better to control location in HASS) in front, but otherwise all seems good.

[larstobi]

@robertsLando Yes, I think you're right.

I'll try to update to the latest dev and see if something else happens.

[robertsLando]

@larstobi ALso If possible I need to see zwavejs2mqtt logs, it's strange

[larstobi]

After updating to yesterdays dev image, I'm not seeing this anymore. Not sure what was fixed in the last three days, but it seems to have worked. :-D

[robertsLando]

Sincerly I have no idea as I never edited nothing in code that could break that :man_shrugging: Glad it's working BTW

[lazyoldbear]

I have seen it again. Sorry.
Interestingly, this time "Rediscover" helped to fix the topics.

[larstobi]

@lazyoldbear What is the "Rediscover" feature? Where can I find it?

[robertsLando]

Rediscover node is a button you find on the top of hass devices table

[lazyoldbear]

I don't know all this smart javascript stuff, but from a brief glance over the source code, here are some silly questions:

• A lot of async puts of this.storeNodes in lib/ZwaveClient.js. The whole object (which is understandable). Although not very likely, it may, theoretically, cause troubles in ordering. E.g. I change the name (it gets updated in the UI, and something else also triggers update of the store, based upon the old data, and coming later, this quietly overrides the node name, whilst UI is unaware). Is there some locking magically in place?
• Same file, similar question. Is it possible that initNode gets called after _setNodeName was triggered, but before it was actually executed?..

I don't know how it is done these days, but strict consistency is a nice thing when it comes to object sets...

[robertsLando]

Is there some locking magically in place?

I cannot think about a case that could cause data loss. I always write the entire object that is always updated, so two consecutive calls even if them are done in the same moment will update it based on the order they comes to js stack, nodejs it's not multi-thread process so there is not this problem

Same file, similar question. Is it possible that initNode gets called after _setNodeName was triggered, but before it was actually executed?..

I'm looking at this too, I saw some users reported me they see unknown manufacturer and product id/type in control panel, I think there could be a case where initNode overrides the node props for some reason. I'm investigating, if it happens to you provide me a log outupt so I can debug this

[lazyoldbear]

I also often saw unknown for Fibaro button. Some back and forth requests of info usually eventually helped.

[lazyoldbear]

Should this be reopened, for there might be a floating trouble?

[robertsLando]

@lazyoldbear I have an idea, let me check

[robertsLando]

233 could be related also to the same problem

[lazyoldbear]

I am just observing the same behaviour as described there. A lot of empty fields. One plug constantly turns dead.
Everything seems fine from HA, though.
This all pretty much tempts to think about loose changing sequences. Async execution has its disadvantages. :-)

In particular, one trouble is that you always store the whole node.
It might be beneficial to decouple UI from the thinking core, and talk to it through an API. So that you don't _setNodeName, but POST/GET that. And process all sequentially on the other end. Easier to summon consistency.
Also it is convenient to run pods separately. One has hardware, persistence and runs the net, another one bells, whistles and UI (and completely stateless).

[robertsLando]

Try with #240

[lazyoldbear]

Recent dev image, same behaviour.

[robertsLando]

@lazyoldbear Yep because you have to try the branch fix-init-node not the dev tag (master branch)

[lazyoldbear]

That might be a bit tricky to test: I don't even have Docker or something similar at home.
But if I get time, I will try to fire it up manually in a blank pod, or push retrieval before the entrypoint.

[lazyoldbear]

Tried. Two fibaro buttons got back with full information. A smart plug is "unknown", but it was dead and I had to revive it.

[robertsLando]

@lazyoldbear Can I close this so?

[larstobi]

@robertsLando I'm seeing this again with the zwavejs/zwavejs2mqtt:dev (2753bf04be41 - 24 hours ago) image.
node_60.json.gz

[robertsLando]

@larstobi Does this only happens on notifications topics? If so that is fixed now

[larstobi]

Thanks, I will try.

[larstobi]

@robertsLando Sorry, still seeing this on docker image zwavejs/zwavejs2mqtt:sha-0767844. :-(

[larstobi]

[robertsLando]

@larstobi This is happening only for discovery right?

[larstobi]

Correct

[robertsLando]

@larstobi Hehehe I was looking at the code without finding any issues. The problem is that your hass devices have "persistent": true,, when persistent is true I don't recreate them but I discover them as they are, it's like a cache system that prevents to override any manual changes on the discovery payload. So please disable the persistent on those and rediscover the node

[larstobi]

What setting am I supposed to disable? I set the "Store" setting under the MQTT settings in zj2m to false. It's description is: "Enable persistent storage of packets (QoS > 0) while client is offline. If disabled the in memory store will be used." Is this the correct one?

Also, I don't seem to have a redisvover button in my Home Assistant under devices, nor under each device.

I did Re-Interview Node 60, and deleted the nodeid_60 topics manually, then set the name of the device in sj2m again. And nodeid_60 showed up in MQTT Explorer right away.

[robertsLando]

@larstobi In hass devices table:

Check the column Persistent, here it says no and it's ok but yours was Yes, it means that you previously pressed on Store button. When it is persistent it means that it's loaded from a cache file (the file nodes.json), for devices loaded from cache the topics are not updated them are sent as them are stored to prevent override the user edits

[larstobi]

I don't have that in my HA. What's the URL? I haven't found a way to navigate there either... I feel like a total n00b...

[robertsLando]

I don't have that in my HA. What's the URL? I haven't found a way to navigate there either... I feel like a total n00b...

LOL that's not on hass, it's on zwavejs2mqtt UI control panel table :)

[larstobi]

LOL, total n00b affiirmed... 😂 😂 😂

Yes, that totally fixed it for me. Thanks for spoon feeding me. 💯 All good now.

[robertsLando]

No worries :)