Core: OpenZWave dimmable modules don't report final state

[ve6rah]

The problem

When using OpenZWave with plugin dimmers or dimmable bulbs, and you turn them "on" or "off" from the UI the following sequence occurs:
1) user toggles switch in UI
2) dimmer starts dimming process
3) Hass queries the dimmer and gets the current state (partially dimmed)
4) Hass updates the frontend with the partially dimmed state
5) dimmer finishes dimming process
6) There is no 6, the frontend is never updated with the final state of the dimmer

Environment

• Home Assistant Core release with the issue: 0.115.0
• Last working Home Assistant Core release (if known): Never with this integration
• Operating environment (OS/Container/Supervised/Core): HassOS 4.13
• Integration causing this issue: OpenZwave
• Link to integration documentation on our website: https://www.home-assistant.io/integrations/ozw/

Problem-relevant configuration.yaml

Traceback/Error logs

Additional information

This is not device specific as I've tried at least 4 different models of dimmer, my current ones are:

• Linear LB60Z-1 Dimmable LED Light Bulb
• Linear PD300Z-2 Plug-in Dimmer Module

In the old Z-Wave integration (https://www.home-assistant.io/docs/z-wave/installation/) this was solved by using device_config with refresh_value and a delay, however I don't see any way to do that on OpenZWave

[MattiasC]

I can add that the Qubino ZMNHDD1 Flush Dimmer is showing the same symptoms.

[FuzzyMistborn]

Have you tried enabling polling?

[MattiasC]

@FuzzyMistborn I have not tried that, not sure how to enable that.

I need to add though, that it would be better to not need to poll for values since that will increase network traffic always, when a single refresh_value would only create one extra read.

[FuzzyMistborn]

https://github.com/OpenZWave/qt-openzwave/blob/master/docs/MQTT.md#enablepoll

I have an automation/script set to run on restart that sends that to every device that I need polling on. I 100% agree that it's not ideal/annoyingly increases traffic but for now it's the only fix I am aware of.

[ve6rah]

@FuzzyMistborn a couple issues there:
1) I looked at the OpenZWave Home Assistant docs. Nowhere does it show polling as an option, nor do I see it anywhere in the WebUI.
2) Polling generates a LOT of extra ZWave (and MQTT?) network traffic for something that should be a matter of simply waiting a few seconds after a change and verifying the result (as was the case in the old Z-Wave integration)

(seems like more of a solution to 40187 than to this bug)

As for your github URL:
1) if this is needed for OpenZWave to function properly, it should be included in the Hass version and/or documentation
2) Looking at that link, I still don't have a clue where to start to actually enable that feature. Can you translate that in to a usable automation that I can understand?

[FuzzyMistborn]

1) OZW is still beta. We just got a UI in .115. Not everything is workable in a UI yet. If that's not acceptable to you, go back to the old integration. In terms of documentation, not everything should be documented in HASS since it's documented in OZW.
1.5) I completely agree with you on the traffic issue, as I said, but the workaround is polling for now since not all devices supported the instant status reporting that is now common among Zwave devices.
2) Here is an example that I have in my script:

    - service: mqtt.publish
      data:
        topic: "OpenZWave/1/command/enablepoll/"
        payload: "{\"ValueIDKey\":1078558737,\"Intensity\":1}"

You need to get the ValueIDKey by using something like MQTT Explorer. Here's a discord conversation about it: https://discord.com/channels/330944238910963714/332357267364249621/744385440756269217

[ve6rah]

@FuzzyMistborn First of all, thank you very much for posting your current workaround (and in a way I can understand), I will be using that method for now until this bug is properly addressed. I will not however be closing this bug, as what you have posted is simply a workaround, not a solution.

I do however have to go on a slight rant here though about your point 1. No offence is intended here, however I see this attitude too often. Telling someone who is submitting a bug report to try to improve an integration that they shouldn't expect usable software because this is just the way beta works is a sure-fire way to never fix the bugs, and never get out of beta. I'm using this new integration, and submitting bugs, because I want to help this get past beta, and be usable for all. Don't belittle my observations as not relevant, work to improve the software!
I will be the judge of whether or not I'm willing to put up with the bugs to help improve things, but attitudes of "don't use it if you don't like it the way it is right now" are not helpful to anyone, and leave the distinct impression that there is no desire to improve and move beyond beta. Observations like that might be helpful in the forums for people who aren't submitting bugs but they are not helpful in bug reports.

Again, I do truly wish to thank you for your workaround. I will be using it until these bugs get fixed.

[FuzzyMistborn]

Sorry if I sounded like I was belittling your observations. I'm fairly certain that adding a button to enable polling is a known issue and something being worked on. My point was that you need to temper your expectations a little (there's nothing wrong with pointing out bugs/issues). I may have mis-interpreted your comments as a complaint, not a constructive comment. Sorry, I just see a lot of "UGH WHY DOESN'T THIS WORK RIGHT NOW."

[ve6rah]

Apology accepted.
I still do not believe that enabling polling is the correct solution here. I think enabling verified changes (I see "changeverified": false in MQTT, would setting that to "true" somehow be helpful?) or replicating the old refresh_value functionality may be more appropriate.
If, as you said, this is an already known issue, please link me to the correct bug and I'll happily close this one as a duplicate. (I didn't find anything in my search)

[FuzzyMistborn]

It's more known based on conversations I've seen in discord in the #zwave channel.

[ve6rah]

@FuzzyMistborn On a side note, is there a way to increase polling frequency? (Yes, I know the risks associated, but it's currently very long, it refreshes each of 5 devices once every 6 seconds, meaning that it takes a full 30 seconds between each poll of a single device)

[FuzzyMistborn]

You could try changing the polling intensity but I haven't done that so I don't know what it does.

[ve6rah]

Polling intensity specifies how often to poll. 1 means poll every cycle, 2 means every second cycle, 3 means every third, etc. 1 is already the lowest, so to go lower you need to reduce the interval between polling cycles.

[probot-home-assistant[bot]]

ozw documentation
ozw source
_{^{(message by IssueLinks)}}

[probot-home-assistant[bot]]

Hey there @cgarwood, @marcelveldt, @martinhjelmare, mind taking a look at this issue as its been labeled with an integration (ozw) you are listed as a codeowner for? Thanks!
_{^{(message by CodeOwnersMention)}}

[nemith]

This seems similar to the issue i reported in #36616

[ve6rah]

Ok, I now have a better workaround. instead of polling the light, we can do a refreshvalue via mqtt after each time we set it.
To accomplish this, I have a node-red flow which listens to the MQTT topic for the light (OpenZWave/1/node/<node_id>/instance/1/commandclass/38/value/<value_id>/), then has a 5 second trigger, and after that a 4 second delay and then publishes {"ValueIDKey":<value_id>} to topic OpenZWave/1/command/refreshvalue/ (5 second trigger just prevents an infinite loop, 4 second delay is the experimental time it takes for the light to fully transition)

[kpine]

I still do not believe that enabling polling is the correct solution here. I think enabling verified changes (I see "changeverified": false in MQTT, would setting that to "true" somehow be helpful?)

If the dimmer's multilevel switch command class is >= V4, then OZW already will refresh the value until it reaches the target value reported by the switch.

If the dimmer's multilevel switch command class is < V4, then yes, you can setting the VerifyChanged compatibility flag to true in the device XML file to force OZW to refresh the value until it reaches the target value. Example here. It works for my GE switch, but there are some quirks (see issue linked below). Since you are using the addon though, it is a difficult workaround because the addon does not provide easy access to the device files. I know some have done it, but as a container user I don't know the details.

In both cases, OZW is polling the value during the transition time. However, the way it's currently implemented results in behavior that is a little erratic, and broken in some cases (some buggy Leviton devices). The issue is still open and in progress. Hopefully this can be addressed when the developer returns from his hiatus.

Note that the above behavior applies to OZW container images after build-150, and includes the addon v0.5.2.

A single refresh value (using mqtt or the zwave integration device configuration) is an OK workaround but falls short when you want to use transitions that are long (e.g. 30 seconds). You either need to globally configure the refresh value to the longest duration you can ever imagine using, or manually issue a refresh as part of an automation or script. It's not a general purpose solution.

Ideally OZW handles this for us, which it is on the path to doing so. If that were not that case, then I would hope that HA would implement a general purpose solution this time.

One minor correction:

1. user toggles switch in UI
2. dimmer starts dimming process
3. Hass queries the dimmer and gets the current state (partially dimmed)
4. Hass updates the frontend with the partially dimmed state
5. dimmer finishes dimming process
6. There is no 6, the frontend is never updated with the final state of the dimmer

In Step 3 replace "Hass" with "OZW". HA is not querying the switch for the dimmer level, OZW is reporting it. By default, OZW does a GET immediately after every SET. In this case the GET is replied to before the dimmer has transitioned to the final level. HA is just reporting the information it gets back from OZW.