Zigbee2mqtt: Delta values for setting bulb brightness / hue / saturation.

[uncle-fed]

While trying to abandon Philips Hue bridge in favour of pure zigbee2mqtt based solution, found an important feature missing.

Philips Hue API supports setting several light's parameters via a relative / delta value as per the following documentation (account/login required): https://developers.meethue.com/develop/hue-api/lights-api/#set-light-state

• brightness: bri_inc (-254 to 254)
• saturation: sat_inc (-254 to 254)
• hue: hue_inc (-65534 to 65534)
• temperature: ct_inc (-65534 to 65534)
• xy: xy_inc (two values supplied as an array of floats with up to 4 decimals)

It would be great to be able to control bulbs in a similar way with zigbee2mqtt.

Not sure if this is easily possible, if the devices do not support delta values directly (you'd then need to know current value, which might lead to a whole new set of possible issues).

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[uncle-fed]

Could we keep this open please?

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[uncle-fed]

Please don't close

[rsalunga29]

@uncle-fed maybe you can follow this tutorial? https://www.zigbee2mqtt.io/how_tos/how_to_support_new_devices.html

[uncle-fed]

@rsalunga29: that tutorial has nothing to do with the topic discussed here.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[uncle-fed]

Bump.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[uncle-fed]

Megabump.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[uncle-fed]

Uberbump

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[uncle-fed]

@Koenkk , can this be considered for v2.0 ? This is the only thing preventing dropping Hue Bridge hardware entirely in favour of pure z2m solution.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Koenkk]

I will look at implementing this some time soon, but first I want to finish among other things #3281

[Koenkk]

I've implemented this for brightness, please let me know if this is the desired behaviour, then I'll continue with the others.

Docs: https://github.com/Koenkk/zigbee2mqtt.io/blob/develop/docs/information/mqtt_topics_and_message_structure.md#zigbee2mqttfriendly_nameset (search for brightness_step).

Changes will be available in the latest dev branch in a few hours (https://www.zigbee2mqtt.io/how_tos/how-to-switch-to-dev-branch.html)

[uncle-fed]

Thanks a lot! I'll check it very soon and report back...

[elfhack]

@Koenkk just put it through it's paces, needs polish

• It seems it's illegal to go from 0 to any value, you can't turn on the bulb with this
• On the flip side, you can't go below 1 if the state is on, eg you can't turn off the bulb with this
• It seems to conflict with the innr bulb hack, where i can get a state of brightness: 1, state: on again when decrementing brightness

However, group attribute propagation seems to work fine :)

[Koenkk]

@elfhack have you also checked brightness_step_onoff? This should allow to turn on/off bulbs.

[elfhack]

Oh sorry, totally missed it in the docs, i'll test it out soon

[elfhack]

Oh right, one more thing - it allows for brightness: 255 to be set, and that value actually sticks in the state. Unless that's a recent change that you made

[Koenkk]

It seems to conflict with the innr bulb hack, where i can get a state of brightness: 1, state: on again when decrementing brightness

Fixed

Brightness 255 is actually fine (waiting for response in https://github.com/Koenkk/zigbee2mqtt/issues/4035#issuecomment-670921397 to confirm).

Changes will be available in the latest dev branch in a few hours (https://www.zigbee2mqtt.io/how_tos/how-to-switch-to-dev-branch.html)

[sjorge]

@Koenkk I can do that for one of my Innr bulbs? It’s en E14 one though... but it also has the same issue. (prob tomorrow as I am not at my apartment today)

[Koenkk]

Yes it works for Inrr (already tested it, found an Innr bulb)

[sjorge]

Brightness 255 is actually fine (waiting for response in #4035 (comment) to confirm).

So this comment no longer need testing?

[Koenkk]

I've already tested that, 255 works fine.

[elfhack]

@Koenkk just gave the brightness_step_onoff a spin, works perfectly for my usecase. Eagerly awaiting the color_temp version too

[uncle-fed]

I've tested brightness_step and brightness_step_onoff today (also in combination with transition time).
Worked fine for me with a classic Philips Hue White & Color Ambiance bulb (9290012573A).
I've also checked the brightness_move which would be very useful too with certain Zigbee switches / scenarios.

I cannot be sure now but once only I managed somehow to turn off the light completely when using brightness_step / brightness_move. I am very sure that it happened with those commands because it was before I tried any of the *_onoff flavours. I am wondering whether somehow I managed to hit brightness value of 1, which is equivalent to turning the bulb off, if I am not mistaken. But no matter what I tried afterwards, I could not reproduce it again. This was all on the Released version, not on the latest dev code.

One minor suggestion, I just would like to propose making the *_onoff feature more prominent on the documentation page.
It is very easy to overlook those commands because they are hidden in the comments (as another tester already confirmed above :-))

[uncle-fed]

OK, I think I may have found it after my "garbage values torture test".
This payload causes my bulb to turn off (exactly like that with the leading zero):

{"brightness_step":-05}

If you specify brightness_step value as a string, then there is another interesting "side effect".
The brightness actually changes but the message that is getting published afterwards in device's topic says that brightness is now null:

# bulb is ON
# published to zigbee2mqtt/bulb/set :   {"brightness_step":"-50"}
# bulb correctly changes brightness
# got this back in zigbee2mqtt/bulb:    {"brightness":null,"state":"ON"}

and another torture, resulting in brightness: 1 reported:

# bulb is ON
# published to zigbee2mqtt/bulb/set:    {"brightness_move_onoff":-05}
# bulb turns OFF instantly
# zigbee2mqtt/bulb reports:             {"state":"OFF","brightness":0}
# query bulb via zigbee2mqtt/bulb/get:  {"brightness":""}
# zigbee2mqtt/bulb reports:             {"brightness":1}

When I try and set other attributes, such as brightness with a string value, there are no issues like that. So it seems like:

• the newly introduced attributes brightness_step / brightness_move do not really enjoy JSON string values and really expect integer (float?), otherwise the brightness is reported as null

• the leading zero on integer value causes bulbs to turn off and set strange values like 'brightness: 1'

[Koenkk]

This payload causes my bulb to turn off (exactly like that with the leading zero): {"brightness_step":-05}

Should be fixed now, since it's not valid JSON this message is not processed anymore and an error will be logged

If you specify brightness_step value as a string, then there is another interesting "side effect".
The brightness actually changes but the message that is getting published afterwards in device's topic says that brightness is now null:

should also be fixed in the latest dev branch in a few hours from now.

[uncle-fed]

More testing here..

I know that there is a different issue opened about this, and that issues looks kinda fixed, but there is still some mess with the 0 and 1 values.

Assuming my understanding is correct that at least for Hue lights (or any Zigbee lights?) brightness = 1, it is the same as brightness = 0 or brightness = null, that is lights turns OFF. So with this in mind, consider the following examples:

• When light is on, publish {"brightness": 1} to z2m/bulb/set.
  The light turns OFF but in the z2m/bulb topic the message is published with {"brightness":1,"state":"ON"}.
  ON is wrong in this case.
  Subsequent query with z2m/bulb/get {"brightness":"", "state":""} returns back values of 1 and OFF.
  It is a bit debatable what should be reported at the end as brightness value (0 or 1) but at least state should not be ON, when the light is OFF.

• Publish {"brightness": 100} to z2m/bulb/set - the light turns ON (if it was off), all is well.
  Now publish {"brightness": 0} to z2m/bulb/set - the light turns OFF and {"brightness":0,"state":"OFF"} is reported.
  Note that here state is correct but brightness is kinda debatable (shouldn't it be 1 according to the previous example?)
  But the value of 0 is not really the actual problem in this example.
  Try z2m/bulb/get {"brightness":"", "state":""} now, while the light is still OFF, and you get back not 0, not 1 but .... 100 !
  I.e. the brightness value that was there before 0 was set (while the state is being reported as OFF, correctly).

• Same example as the previous one, but instead of 0, set the brightness to null.
  Light goes OFF, brightness reported as 0, state as OFF but just as above, the subsequent get query returns brightness = 100.

• Now one more torture test. Same as the last two, but instead of 0 or null, set brightness value to a non numeric string.
  Publish {"brightness": "some crap"} to z2m/bulb/set while the light is on.
  Light goes OFF, but report comes back as {"brightness":null,"state":"ON"}, i.e. total mess.
  The subsequent get query returns, surprisingly, not null and not 0but {"brightness":1,"state":"OFF"} !

OK this was about ordinary brightness and now to the new attributes such as brightness_step and brightness_move.

• Set bulb brightness to 2, the bulb is ON.
  Publish {"brightness_step":-1} to z2m/bulb/set... what is the expected behaviour?
  I think, if we still under the assumption that 1 is equivalent to "OFF" state, and we're NOT using brightness_step_onoff, then the expected behaviour should be that brightness remains at 2.
  Instead it goes to 1 and happily reports this as {"brightness":1,"state":"ON"}, which contradicts to all of the above examples, where 1 means OFF.

• Publish some more garbage to the set topic: {"brightness_step":"more_crap"}
  Nothing changes, which is good.. but the reported values are: {"brightness":null,"state":"ON"}.
  Not really ideal... the command should not be processed rather, same way how you would reject value like -999.

• And speaking of "out of acceptable range" values, why does brightness reject value like -9, saying that the acceptable values "must be >= 0 and <= 255", but then it happily accepts a value of 12345 (setting it to 254 maximum possible value). Why is the range declared <=255 ? It should be no upper limit declared then. And if that upper boundary value is there to say that the max _sensible_ value is 255, then it is also wrong, because it is actually 254 :-))

[Koenkk]

Assuming my understanding is correct that at least for Hue lights (or any Zigbee lights?) brightness = 1, it is the same as brightness = 0 or brightness = null, that is lights turns OFF. So with this in mind, consider the following examples:

Unfortunately not, Hue/Inrr lights indeed seem to turn off, but e.g. TRADFRI stays on. For Zigbee2MQTT, brightness 0 == OFF, anything else is ON.

Fixed the issues you mentioned.

About:

And if that upper boundary value is there to say that the max sensible value is 255, then it is also wrong, because it is actually 254 :-))

Now an error will be logged when the value >= 256, or < 0: 'Error: Brightness value of message: '{"brightness":256}' invalid, must be a number >= 0 and =< 254', I silently allow 255 on purpose to prevent a breaking change (as probably a lot of people use this value to set the max brightness).

Changes will be available in the latest dev branch in a few hours (https://www.zigbee2mqtt.io/how_tos/how-to-switch-to-dev-branch.html)

[uncle-fed]

Slowly getting there.. :-) But not quite..

publish z2m/bulb/set {"brightness":100} (bulb turns ON)
publish z2m/bulb/set {"brightness":0} (bulb turns off and {"brightness":0,"state":"OFF"} is reported)
publish z2m/bulb/set {"brightness_step":0}
1st wrong behaviour: report comes in z2m/bulb that {"brightness":1,"state":"ON"} (the light is still off BTW)
immediately query bulb via z2m/bulb/get {"brightness":""}
2nd wrong behaviour: report comes now that {"brightness":100} (light is still off)

Also, still did not fully get what is z2m's concept regarding Hue lights and brightness 1.
Does z2m treat all kind of lights equally and insists that 1 = ON for all lights?
How does it make sense that z2m reports state as ON, while in reality device can be OFF, when brightness is 1.

And it wouldn't be such a major thing and one could make a disclaimer that people should avoid using "ambiguous" value of 1 for Hue lights, if it wasn't for the fact that sometimes it won't be possible to avoid this value because it can come as a result of _step or _move.

Here is an example that demonstrates why the concept "is very strange" currently, to say the least:

publish z2m/bulb/set {"brightness":100} (bulb turns ON)
publish z2m/bulb/set {"brightness_move": -20} (bulb fades to its lowest brightness level, supposedly)
read the current brightness level, publish z2m/bulb/get {"brightness":""}
reported brightness: 1, and light is still ON.
so.. if I set the same brightness as there is currently, nothing should change, right?
publish z2m/bulb/set {"brightness":1} (bulb changes its state to OFF, but brightness remains reported at 1)

So basically, to sum it up,

• currently, z2m allows the bulb to be in both OFF and ON state while the brightness is reported as the same value of 1
• currently, z2m can report bulb being ON while it is OFF depending how you came to a value of 1

One other thing, maybe my eyes deceive me, but I've checked it multiple times and with my particular Hue bulb here it looks like when I use "brightness_move" with negative value, the bulb slowly fades to minimum brightness but does not stop and goes a tiny bit brighter again before it stops changing brightness level. This was when I used -50 as a value.

P.S. It looks like the commit 3c3ed3c did not pass some checks and has a cross near it. Does it mean that I was testing the code that you did not intend to be tested? ;-)

[Koenkk]

publish z2m/bulb/set {"brightness":100} (bulb turns ON)
publish z2m/bulb/set {"brightness":0} (bulb turns off and {"brightness":0,"state":"OFF"} is reported)
publish z2m/bulb/set {"brightness_step":0}
1st wrong behaviour: report comes in z2m/bulb that {"brightness":1,"state":"ON"} (the light is still off BTW)
immediately query bulb via z2m/bulb/get {"brightness":""}
2nd wrong behaviour: report comes now that {"brightness":100} (light is still off)

This should be fixed now.

Also, still did not fully get what is z2m's concept regarding Hue lights and brightness 1.
Does z2m treat all kind of lights equally and insists that 1 = ON for all lights?
How does it make sense that z2m reports state as ON, while in reality device can be OFF, when brightness is 1.

We don't treat all lights equally, specific code is enabled by setting https://github.com/Koenkk/zigbee-herdsman-converters/commit/3c3ed3c62af7d851963d76c019138659800d92d0#diff-6c9a6acf22f90d1c6e524d9f3c5c1745R2039. However I accidentally reverted everything in https://github.com/Koenkk/zigbee-herdsman-converters/commit/2a8513b720c296350411244c3c619b11ef6ab239 which may be the reason why you are not seeing any changes. With this we try to provide the same behaviour for all bulbs, brightness 0 is off, anything else is on. It would be annoying if this wasn't the case, I feel Zigbee2MQTT is the layer that should abstract this.

For bulbs having turnsOffAtBrightness1: true, when setting {"brightness":1}, the expected behaviour is that the bulb stays ON and that the returned brightness is 2 (as with 1 it would turn off). When doing a get, it should also return brightness 2.

However, I noticed some inconsistency in the bulb itself, brightness 1 is not always off! E.g.: Set brightness 2, brightness_step -1, get brightness -> bulb returns brightness 1 and it is still ON.

Changes will be available in the latest dev branch in a few hours (https://www.zigbee2mqtt.io/how_tos/how-to-switch-to-dev-branch.html)

[uncle-fed]

Thanks for the explanations and fixes, so far looks good, I've tested around and can see that brightness = 1 does not cause lights being OFF anymore.

Perhaps one more irregular behaviour (assume light is at full brightness at the start of each command):

{"state":"OFF", "transition":3} - takes 3 seconds to gradually fade away to OFF state
{"brightness":1, "transition":3} - also takes 3 seconds to gradually fade away to lowest brightness
{"brightness":0, "transition":3} - light fades and turns off using default Hue routine (i.e. not instantly but definitely much faster than 3 seconds)

[Koenkk]

@uncle-fed you are very good at finding bugs :) fixed it!

Changes will be available in the latest dev branch in a few hours (https://www.zigbee2mqtt.io/how_tos/how-to-switch-to-dev-branch.html)

[uncle-fed]

Thanks very much. I think I am now ready for more testing of saturation and hue values ;-)

[Koenkk]

I've added the following commands: color_temp_step, hue_step, saturation_step, saturation_move and hue_move.

Docs (I've unified it a bit): https://github.com/Koenkk/zigbee2mqtt.io/blob/develop/docs/information/mqtt_topics_and_message_structure.md

Changes will be available in the latest dev branch in a few hours (https://www.zigbee2mqtt.io/how_tos/how-to-switch-to-dev-branch.html)

[uncle-fed]

Something strange... I can use saturation_move and hue_move with my Philips bulb (https://www.zigbee2mqtt.io/devices/9290012573A.html) but simple saturation or hue says: "no converter available"....

[sjorge]

Might just not be hooked up yet, I had to do some extra work to get them working for Innr bulbs which are nearly feature compatible with Hue.

[uncle-fed]

but... I thought simple hue / saturation worked for a long time now with Hue bulbs and this issue here is for the new features that send delta values. Right now only delta values work but not ordinary setting of 'hue' or 'saturation'. I am confused...

[Koenkk]

@uncle-fed {"color": {"hue": XX, "saturation": XX}} should help :)

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days

[uncle-fed]

Was unable to test due to lack of time but will resume shortly.