Zigbee2mqtt: Support track direction: Xiaomi Aqara Smart Curtain Motor ZNCLDJ11LM

[yozik04]

We have support of Xiaomi Aqara Smart Curtain Motor. But we need a "reverse" mode as well.

I have a problem. I have 3 motors. If all are physically closed, 2 of them reports that they are open and one closed.

How hard it is to implement reverse mode? Should it be done in zigbee2mqtt or in HomeAssistant?

Initial Xiaomi Aqara Smart Curtain Motor support was introduced in #294

[yozik04]

I figured out how to get device specific configuration from yaml in fromZigbee.js.
But I do not understand how to get it in toZigbee.js.

[Koenkk]

What exactly do you want to add to toZigbee.js, also what is this "reverse mode"?

[yozik04]

Curtains can open to the left or to the right. So open state is different.
2 of my curtains open to the left. Which is reverse. It causes motor to spin in the opposite direction.
In MiHome there is a switch to do this.

[yozik04]

[yozik04]

In _toZigbee.js_ ZNCLDJ11LM_control method I would like to get device specific setting called "reverse" or "direction". In case of "positive" direction:

                const lookup = {
                    'open': 100,
                    'close': 0,
                    'on': 100,
                    'off': 0,
                };

in case of "reverse" direction:

                const lookup = {
                    'open': 0,
                    'close': 100,
                    'on': 0,
                    'off': 100,
                };

Makes sense?

[Koenkk]

I see, I think direction makes the most sense and then with positive (default) and reverse as the options (basically just what is shown in the Xiaomi app.

[yozik04]

Let's use direction then. But how to get this setting in toZigbee.js ZNCLDJ11LM_control method? =)

[Koenkk]

You can add an extra parameter to the convert function called options, you need to pass it here: https://github.com/Koenkk/zigbee2mqtt/blob/dev/lib/extension/devicePublish.js#L171 and https://github.com/Koenkk/zigbee2mqtt/blob/dev/lib/extension/devicePublish.js#L205. The deviceSettings are retrieved here https://github.com/Koenkk/zigbee2mqtt/blob/dev/lib/extension/devicePublish.js#L202

[dzungpv]

@yozik04 just sniff packet with original hub when you change the direction, and then add to toZigbee code i remember it send a command and store in the motor memory, but i sniff it long time ago and then not save the packet.

[yozik04]

I do not have original hub =(

[yozik04]

@Koenkk I see you have added options in 10.0.0. Do I get it right that it serves different purpose than deviceSettings?

[yozik04]

@Koenkk I think you have mistake there:
https://github.com/Koenkk/zigbee2mqtt/blob/7cabe3a5234edfd52029308a2e3fff22b1a8e46b/lib/extension/devicePublish.js#L177

https://github.com/Koenkk/zigbee2mqtt/blob/7cabe3a5234edfd52029308a2e3fff22b1a8e46b/lib/extension/devicePublish.js#L211

In second postfix is missing before options

[Koenkk]

@yozik04 thanks, fixed!

[Koenkk]

@dzungpv would you mind sniffing setting the track direction?

[dzungpv]

@Koenkk i sniff the packet:

Some thing like this need to test with your side @yozik04 , add to toZigbee.js, because i have much device, must run xiaomi hub, not test with zigbee2mqtt. When i capture datatype is string: 0x42, but string not write raw hex if i remember correctly with z2m, it will convert to ASCII code, so let try uint16 0x21

    ZNCLDJ11LM_direction: {
        key: ['direction'],
        convert: (key, value, message, type, postfix, options) => {
            const cid = 'genBasic';

            if (type === 'set') {
                const lookup = {
                    'positive': 0x0100000000070007,
                    'reverse': 0x0100000101070007,
                };

                if (lookup.hasOwnProperty(value)) {
                    return [{
                        cid: cid,
                        cmd: 'write',
                        cmdType: 'foundation',
                        zclData: [{
                            attrId: 0x0401, // presentValue
                            dataType: 0x21, // dataType
                            attrData: lookup[value],
                        }],
                        cfg: cfg.xiaomi,
                    }];
                }
            } else if (type === 'get') {
                return [{
                    cid: cid,
                    cmd: 'read',
                    cmdType: 'foundation',
                    zclData: [{
                        attrId: 0x0401
                    }],
                    cfg: cfg.xiaomi,
                }];
            }
        },
},

[yozik04]

Trying:
Do I need to add something to fromZigbee.js ?

[yozik04]

zigbee2mqtt          | RangeError [ERR_OUT_OF_RANGE]: The value of "value" is out of range. It must be >= 0 and <= 65535. Received 72057598350131200
zigbee2mqtt          |     at checkInt (internal/buffer.js:35:11)
zigbee2mqtt          |     at writeU_Int16LE (internal/buffer.js:543:3)
zigbee2mqtt          |     at Buffer.writeUInt16LE (internal/buffer.js:551:10)
zigbee2mqtt          |     at Concentrate.write_number (/app/node_modules/zigbee-herdsman/node_modules/concentrate/index.js:63:21)
zigbee2mqtt          |     at Concentrate.<anonymous> (/app/node_modules/zigbee-herdsman/node_modules/concentrate/index.js:42:29)
zigbee2mqtt          |     at Array.forEach (<anonymous>)
zigbee2mqtt          |     at Concentrate.result (/app/node_modules/zigbee-herdsman/node_modules/concentrate/index.js:38:13)
zigbee2mqtt          |     at getDataTypeBuf (/app/node_modules/zigbee-herdsman/dist/zcl-packet/lib/foundation.js:463:24)
zigbee2mqtt          |     at /app/node_modules/zigbee-herdsman/dist/zcl-packet/lib/foundation.js:141:46
zigbee2mqtt          |     at Array.forEach (<anonymous>)

Probably dataType is wrong.

[yozik04]

0x0100000000070007 is 8 byte

[dzungpv]

@yozik04 do your last commit work?

[yozik04]

It successfully sends. But no change to the behaviour.

Zigbee publish to device 'living_room_right_curtain', genBasic - write - [{"attrId":1025,"dataType":66,"attrData":"\u0001\u0000\u0000\u0001\u0001\u0007\u0000\u0007"}] - {"manufSpec":1,"disDefaultRsp":1,"manufCode":4447} - 1

[dzungpv]

@yozik04 As i said from the start, this string write currently not work with z2m, if you use 0x42 then zcl-packet will convert it to ASCII string and not the original. May be we will find other way to write raw data (hexstream) @Koenkk ?
Also other packet capture here can help us:
https://github.com/TedTolboom/com.xiaomi-mi-zigbee/blob/master/drivers/curtain/device.js

[dzungpv]

@yozik04 let try this:

                const lookup = {
                    'positive': '\u0001\u0000\u0000\u0000\u0000\u0007\u0000\u0007',
                    'reverse': '\u0001\u0000\u0000\u0001\u0001\u0007\u0000\u0007',
                };

Or

                const lookup = {
                    'positive': '\u01\u00\u00\u00\u00\u07\u00\u07',
                    'reverse': '\u01\u00\u00\u01\u01\u07\u00\u07',
                };

with datatype 0x42

[yozik04]

@dzungpv first is equal to 'x01x00x00x01x01x07x00x07' form
second is wrong syntax.

[dzungpv]

@yozik04 i see the last commit, it work or not?

[yozik04]

It successfully sends. But no change to the behaviour.

Zigbee publish to device 'living_room_right_curtain', genBasic - write - [{"attrId":1025,"dataType":66,"attrData":"\u0001\u0000\u0000\u0001\u0001\u0007\u0000\u0007"}] - {"manufSpec":1,"disDefaultRsp":1,"manufCode":4447} - 1

No change. Motors keep working in the same direction.

[Koenkk]

@yozik04 can you verify if the correct message is send by sniffing the packages? (http://www.zigbee2mqtt.io/how_tos/how_to_sniff_zigbee_traffic.html) Should be the same as https://github.com/Koenkk/zigbee2mqtt/issues/1639#issuecomment-506706135

[yozik04]

Do not have second Zigbee module by my hands. Will try to find... Be patient =)

[yozik04]

No, it writes just a first byte from a string.

[yozik04]

Maybe I was on old zigbee2mqtt version.
https://github.com/Koenkk/zigbee-herdsman/commit/bc265563aa9fecc13f2356a8b5fa8fd65dcb7e9e

Should this help?

[Koenkk]

@yozik04 that won't help probably

@dzungpv with what did you sniff? In https://github.com/Koenkk/zigbee2mqtt/issues/1639#issuecomment-506706135 it shows the datatype as being a string, however two lines below it shows unknown data type, therefore I don't know how to read this. Could you sniff it with wireshark?

[yozik04]

One more thing about these Xiaomi Curtains.

Strange MQTT publish duplication in logs:
https://pastebin.com/9BVqKk7M

Last publish is also wrong. Position should be 100 in the end. Not null. Null is only when curtain is moving.

[dzungpv]

@Koenkk I will try with Wireshark

[yozik04]

No luck?

[freenetwork]

May be so:

const ZNCLDJ11LM_direction = {
        key: ['direction'],
        convert: (key, value, message, type, postfix, options) => {
            const cid = 'genBasic';

            if (type === 'set') {
                const lookup = {
                    'positive': '\x01\x00\x00\x00\x00\x07\x00\x07',
                    'reverse': '\x01\x00\x00\x01\x01\x07\x00\x07',
                };

                if (lookup.hasOwnProperty(value)) {
                    return [{
                        cid: cid,
                        cmd: 'write',
                        cmdType: 'foundation',
                        zclData: [{
                            attrId: 0x0401, // presentValue
                            dataType: 0x42, // dataType
                            attrData: lookup[value],
                        }],
                        cfg: {
                            manufSpec: 1,
                            disDefaultRsp: 1,
                            manufCode: 0x115F,
                        },                    }];
                }
            } else if (type === 'get') {
                return [{
                    cid: cid,
                    cmd: 'read',
                    cmdType: 'foundation',
                    zclData: [{
                        attrId: 0x0401
                    }],
                    cfg: {
                        manufSpec: 1,
                        disDefaultRsp: 1,
                        manufCode: 0x115F,
                    },
                }];
            }
        },
}

[freenetwork]

I'm already dizzy, but it seems to work

[yozik04]

Did you tried to make a capture with Wireshark? Does output look good?

[freenetwork]

I does not have hub. I have cc2531 and deconz and aqara b1 motor. And i have cdr radio. I try capture traffic.

UPD: i buy hub and today test capture message from zigbee2mqtt to curtain controller
UPD2: i capture messages for aqara b1 motor. I cant upload files. It is in my repo

[yozik04]

From @freenetwork captures:
set reverse mode:

set direct mode:

[yozik04]

I have tried it with my curtain motor. Response is: Unsupported Attribute.

[yozik04]

@freenetwork Your dump is for ZNCLDJ12LM which are with battery. Not for ZNCLDJ11LM that we discuss here.

[yozik04]

@dzungpv Can you still do the capture for your motors?

A good howto:
https://www.zigbee2mqtt.io/how_tos/how_to_sniff_zigbee_traffic.html

[dzungpv]

@yozik04 I think Wireshark or Ubiqua not difference in packet format, some employer of TI has tool to convert Wireshark packet to Ubiqua too, So the packet is correct but z2m not fully support write to it.
With new https://github.com/Koenkk/zigbee-herdsman support in version 1.6 dev, we will re try later

[yozik04]

Sometimes when my curtain looses connection to the zigbee2mqtt bridge I get this weird behaviour:
When curtain gets back online it automatically opens. See log:

Isn't it a bug that genAnalogOutput writes "attrData":null ?
Also for some reason MQTT publish: topic messages are duplicated.

9/21/2019, 7:13:25 PM - error: Failed to ping 'living_room_left_curtain'
9/21/2019, 7:13:25 PM - info: MQTT publish: topic 'zigbee2mqtt/living_room_left_curtain/availability', payload 'offline'
9/21/2019, 7:14:25 PM - info: MQTT publish: topic 'zigbee2mqtt/living_room_left_curtain/availability', payload 'online'
9/21/2019, 7:14:25 PM - info: MQTT publish: topic 'zigbee2mqtt/living_room_left_curtain/availability', payload 'online'
9/21/2019, 7:14:25 PM - info: Zigbee publish to device 'living_room_left_curtain', genAnalogOutput - write - [{"attrId":85,"dataType":57,"attrData":null}] - {"manufSpec":0,"disDefaultRsp":0} - 1
9/21/2019, 7:14:26 PM - info: MQTT publish: topic 'zigbee2mqtt/living_room_left_curtain', payload '{"position":100,"running":true,"linkquality":23}'
9/21/2019, 7:14:26 PM - info: MQTT publish: topic 'zigbee2mqtt/living_room_left_curtain', payload '{"position":null,"running":true,"linkquality":23}'

[Koenkk]

Probably the value received is not in the lookup (https://github.com/Koenkk/zigbee-shepherd-converters/blob/master/converters/toZigbee.js#L1199). Can you provide the same logging with debug enabled?

To enable debug logging set in configuration.yaml:

advanced:
  log_level: debug

[yozik04]

Want to ask same question after couple months: Does zigbee-herdsman now support sending a Character string to a device?

[Koenkk]

Looking back, I think we are misinterpreting the logs from (at least I don't understand how to read it). https://github.com/Koenkk/zigbee2mqtt/issues/1639#issuecomment-506706135, it shows an attribute within an attribute??? For me in order to continue on this I would need to have the wireshark logs.

[yozik04]

Reverse:

Original:

[yozik04]

Some kind of reset move command (When an obstacle was removed):

Just in case we want it some day. Actually it is useful for me for my one glitchy motor. I have to reset it every 2-4 weeks. It just stops working.

[yozik04]

Some more 😃:
Disable opening by hand:

Enable opening by hand:

[yozik04]

Summary:
If opening by hand is enabled and I change direction to original
07 00 02 00 00 04 00 12
If opening by hand is enabled and I change direction to reverse
07 00 02 00 01 04 00 12
If opening by hand is disabled and I change direction to original
07 00 02 00 00 04 01 12
If opening by hand is disabled and I change direction to reverse
07 00 02 00 01 04 01 12

If direction is reverse and I change opening by hand to off
07 00 08 00 01 04 01 12
If direction is reverse and I change opening by hand to on
07 00 08 00 01 04 00 12
If direction is original and I change opening by hand to off
07 00 08 00 00 04 01 12
If direction is original and I change opening by hand to on
07 00 08 00 00 04 00 12

If opening by hand is on and direction is original and I clear move
07 00 01 00 00 04 00 12
If opening by hand is off and direction is original and I clear move
07 00 01 00 00 04 01 12
If opening by hand is off and direction is reverse and I clear move
07 00 01 00 01 04 01 12
If opening by hand is on and direction is reverse and I clear move
07 00 01 00 01 04 00 12

If direction is reverse and I click close curtains

If direction is reverse and I click open curtains

If direction is original and I click close curtains

If direction is original and I click open curtains

[yozik04]

Some numeric percentage (20%) when in reverse direction

Some numeric percentage (20%) when in original direction

Stop command:

[Koenkk]

Could you share the pcapng (wireshark) files?

[yozik04]

I will have to make new ones. I did very many and did not saved anything. Is anything missing from the screensots?

[Koenkk]

I want to look at the exact bytes send (once you click on the value), therefore having these would be the easiest (otherwise it is guessing for me).

[yozik04]

I've sent capture to your github email.

[Koenkk]

Thanks, I've made some necessary modification to zigbee-herdsman in order to support this.

First make sure you update to the latest dev.

I've created an example changeset on how to write a certain feature: https://github.com/Koenkk/zigbee-herdsman-converters/commit/c2b4b167e6951b573423e366d62c4193b5b98b5a (probably this is not the correct payload for reverse, this is just an example)

Looking at your screenshots, the only thing you have to change each time is the payload: https://github.com/Koenkk/zigbee-herdsman-converters/commit/c2b4b167e6951b573423e366d62c4193b5b98b5a#diff-3e0c275bd41af699153bf59d8dda4a2aR752

When for example sniffing the following write request:

You click on the String: which will highlight the bytes in blue below. After this add each byte postfixed with 0x and put this in the payload. So for this example it would be: const payload = [0x07, 0x00, 0x02, 0x01, 0x00, 0x00, 0x14];.

Now you can trigger the command by sending to zigbee2mqtt/[FRIENDLY_NAME]/set with payload {"reverse": ""}. Where reverse is the value in key of the toZigbee converter.

[yozik04]

Is it possible in convertSet to get previous 8 byte value?

[yozik04]

Can I actually do differently? I want to name the property to options or settings as all options are stored in one zigbee attribute.

mqtt zigbee2mqtt/[FRIENDLY_NAME]/set payloads to set options will be:

• {"options": {"direction":"original","hand_open":true,"reset_move":false}} default
• {"options": {"direction":"reverse"}} direction changed, hand_open resets
• {"options": {"hand_open":false}} direction resets, hand_open is disabled
• {"options": {"reset_move":true}} reset triggered, direction and hand_open changed to default

from these I will generate the right payload.

[Koenkk]

Yes that is possible, change the key to options and parse the value as json.

[yozik04]

Good, thank you for help! Will try to implement and test it this week.

[bernardyeo]

Tried making the changes manually to devices.js and toZigbee.js. Ended up with the following error. Running on 1.8.0.

zigbee2mqtt:error 2019-12-16 23:41:22: Publish 'set' 'reverse' to '0x00158d00042b0ed3' failed: 'TypeError: argument must be a string'

Sent payload of {"reverse": ""} to zigbee2mqtt/[FRIENDLY_NAME]/set. I am unsure if I did anything wrong.

[yozik04]

@Koenkk Which package I should update to the latest dev?

When setting:

error 2019-12-20T14:49:01: Publish 'set' 'options' to 'living_room_left_curtain' failed: 'TypeError: argument must be a string'
info  2019-12-20T14:49:01: MQTT publish: topic 'zigbee2mqtt/bridge/log', payload '{"type":"zigbee_publish_error","message":"Publish 'set' 'options' to 'living_room_left_curtain' failed: 'TypeError: argument must be a string'","meta":{"friendly_name":"living_room_left_curtain"}}'

When getting:

error 2019-12-20T14:57:22: Publish 'get' 'options' to 'living_room_left_curtain' failed: 'Error: Cluster 'genBasic' has no attribute '1025''
info  2019-12-20T14:57:22: MQTT publish: topic 'zigbee2mqtt/bridge/log', payload '{"type":"zigbee_publish_error","message":"Publish 'get' 'options' to 'living_room_left_curtain' failed: 'Error: Cluster 'genBasic' has no attribute '1025''","meta":{"friendly_name":"living_room_left_curtain"}}'

Code here: https://github.com/yozik04/zigbee-herdsman-converters/tree/ZNCLDJ11LM_direction

[yozik04]

I managed to make it send. But it does not send Manufacturer code and device responds with 'Unsupported attribute'

[yozik04]

Another problem after update of my docker container to latest-dev I lost all my devices. 😢

[Koenkk]

As this is now supported, I assume this can be closed, thank you very much @yozik04 !!

[bernardyeo]

@Koenkk I updated zigbee-herdsman-converters to the latest version but encountered the following error when I attempt to set reverse_direction to true:

zigbee2mqtt:info 2020-01-03 22:47:38: ZNCLDJ11LM setting reverse direction
zigbee2mqtt:error 2020-01-03 22:47:38: Publish 'set' 'options' to '0x00158d00042b0ed3' failed: 'TypeError: argument must be a string'
zigbee2mqtt:info 2020-01-03 22:47:38: MQTT publish: topic 'zigbee2mqtt/bridge/log', payload '{"type":"zigbee_publish_error","message":"Publish 'set' 'options' to '0x00158d00042b0ed3' failed: 'TypeError: argument must be a string'","meta":{"friendly_name":"0x00158d00042b0ed3"}}'

[yozik04]

@bernardyeo you need to update other components too.

@Koenkk Thank you for your great support!

[bernardyeo]

@yozik04 Thanks I will give it a try.