Controllerx: zha support

[sreknob]

Firstly, thanks for this very cool project!
I see that you recently added Deconz support and I was just wondering if you would consider zha support as well.
Seems like you already have some framework for event listening for Deconz.
Zha puts out events on zha_event with a device_ieee (instead of event_id) issues commands with args as necessary.
For example, for a ICTC-G-1 an event is this:

{
    "event_type": "zha_event",
    "data": {
        "unique_id": "00:0b:57:ff:fe:2c:37:f0:1:0x0008",
        "device_ieee": "00:0b:57:ff:fe:2c:37:f0",
        "endpoint_id": 1,
        "cluster_id": 8,
        "command": "move_with_on_off",
        "args": [
            0,
            195
        ]
    },
    "origin": "LOCAL",
    "time_fired": "2020-01-22T01:14:32.409810+00:00",
    "context": {
        "id": "0bc078cec3c34adcb9d0a3d11b9629ac",
        "parent_id": null,
        "user_id": null
    }
}

Not sure how much work this would be, but thanks for considering it!

[xaviml]

Hi @sreknob,

Thank you for your interest in this project. I had in mind to givr support to ZHA as well. I am waiting for a ConBee II device to arrive and I will be able to test it myself.

However, if you volunteer to test the device that you have, I can start developing for it. Let me know if rhe following is correct:

• "event_type" is always zha_event
• "device_ieee" is the device you pressed the button. If so, is there any field that indicates a more friendly name?
• "command" is the button that was pressed on the device beforementioned.

Thanks :)

[sreknob]

That's awesome that you're willing! Thank you. You're already well on your way to creating the swiss army knife of appdaemons for zigbee controllers :-)
Unfortunately, there are no friendly names in zha. I believe this is by design of zha to keep it stateless and as universal as possible so that there doesn't have to be a quirk/driver for every device, only those that need custom mappings/non-compliant with zigbee standards need quirks.
The upstream quirks library for ikea remotes is at https://github.com/dmulcahey/zha-device-handlers/tree/dev/zhaquirks/ikea (with other devices in the repo as well). Alexi and David who are the primary maintainers for zha and quirks for HA are super friendly and helpful guys too.
I'm actually going away on vacation until next weekend but will get you the commands and args for the devices I have when I get back!
Many thanks.

[xaviml]

I am doing some refactoring to the project and I am taking into account the integration for ZHA. However, I observed some things that are different from how deconz and z2m works. I will contact the maintainers of the zha you mentioned and I will keep you updated with it.

Thanks for pointing me in the right direction :)

[mck182]

I'd also love to see ZHA support!

I'm happy to help test this with my Ikea stuff, although I'm still new to this world, so I'd need a bit of handholding :)

[xaviml]

Thanks @mck182 :) I will keep you updated with instructions for it! Which IKEA controllers do you have?

[mck182]

Right now only the E1743 switch/dimmer. I also have the E1745 motion sensor but I guess that's out of the scope for this. Plus a bunch of Ikea bulbs and countertop lights.

[xaviml]

Hi @mck182 and @sreknob,

I gave support to ZHA and the first device I could integrate was E1743. My setup is ZHA with deconz and the button is able to turn on/off and bright and dim the light(s). These are the instructions to try the code:

Through HACS

If you go to ControllerX in HACS, you can select in the menu Show Beta and install v2.2.0b1. Then you will need to restart your AppDaemon server.

Through files

You can download the dev branch and copy the controllerx folder inside apps into /config/appdaemon/apps/. Then you will need to restart your AppDaemon server.

If you try it, let me know about it. Thanks :)

[mck182]

Ooh awesome, thanks! I was waiting with installing AppDaemon till I can test with something, but now I'm stuck installing it - apparently the docker on Raspberry Pi has some issues which prevent building the docker image for AppDaemon. I'm looking into resolving that and then I'll test this right away!

[sreknob]

Wow, this is great @xaviml

Have you figured out how to process the args to distinguish between the left and right button presses, etc coming from zha? I don't see anything in the beta repo files about it yet.

For the 5-button remote (E1524 controller) this adds basic functionality for zha. This includes toggle, brightness up/down and cycle through colors.

Unfortunately, until there are ways to distinguish between left and right arrows, they both do the same thing.

    def get_zha_actions_mapping(self):
        return {
            # centre press
            "toggle": self.toggle,
            # brightness_up_click
            "step_with_on_off": (
                self.click,
                LightController.ATTRIBUTE_BRIGHTNESS,
                Stepper.UP,
            ),
            # brightness_down_click
            "step": ( # 
                self.click,
                LightController.ATTRIBUTE_BRIGHTNESS,
                Stepper.DOWN,
            ),
            # arrow_left_click AND arrow_right_click --> can only tell via args
            "press": (
                self.click,
                LightController.ATTRIBUTE_COLOR,
                Stepper.UP,
            ),
            # brightness_up_hold
            "move_with_on_off": (
                self.hold,
                LightController.ATTRIBUTE_BRIGHTNESS,
                Stepper.UP,
            ),
            # brightness_down_hold
            "move": (
                self.hold,
                LightController.ATTRIBUTE_BRIGHTNESS,
                Stepper.DOWN,
            ),
            # brightness_up_release AND brightness_down_release
            "stop": self.release,
            # arrow_left_hold AND arrow_right_hold --> can only tell via args
            "hold": (
                self.hold,
                LightController.ATTRIBUTE_COLOR,
                Stepper.DOWN,
            ),
            # arrow_left_release AND arrow_right_release
            "release": self.release,
        }

Also, I think for E1743 controller, the zha section should be Stepper.UP for move_with_on_off instead of Stepper.TOGGLE --> I've made this change and it works as expected for the up hold now.

And also one more thing, as controllers are getting support for deconz, zha, etc. perhaps it would be good to have a column in https://github.com/xaviml/controllerx/wiki/Supported-controllers that lists what integration it supports.

I have the ICTCG1 as well, so I will look at that next! :-)

[xaviml]

Hi @sreknob

Thanks for trying it out. I was working right now on it. I just uploaded the dev branch with the latest code. I process the command and args from the event and I created an action with it. For example, if the command is move and the args are [1,83], then a string move_1_83 is created and that is how you map for the ZHA devices. I had to ignore arguments for release command since apparently it sends the time has been held and it's not a fixed value.

Now I gave support to E1810 (Light and media player), symfonisk controller and E1743 (I fixed the Stepper.TOGGLE). I tried them all out and it works like a charm. Let me know if you try it out :)

Regarding the documentation, I think it's a good idea to add the integration supported for the device. I am also working on changing the documentatin to Github pages and organize it a little bit better.

If you want to give support to the ICTCG1, you can let me know about the events that the controller sends or you can create a PR and I will add it.

Thanks you for your collaboration :)

[xaviml]

I created a new pre-release, you can download it from HACS as well, so you don't have to copy and paste anything.

[mck182]

Ok so I got AppDaemon working, but I can't seem to successfully run this app. Here's the log:

2020-02-02 23:44:58.908399 WARNING livingroom_controller: ------------------------------------------------------------
2020-02-02 23:44:58.908953 WARNING livingroom_controller: Unexpected error running initialize() for livingroom_controller
2020-02-02 23:44:58.909462 WARNING livingroom_controller: ------------------------------------------------------------
2020-02-02 23:44:58.912070 WARNING livingroom_controller: Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/appdaemon/app_management.py", line 144, in initialize_app
    await utils.run_in_executor(self, init)
  File "/usr/local/lib/python3.8/site-packages/appdaemon/utils.py", line 282, in run_in_executor
    response = future.result()
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/conf/apps/controllerx/core/type/light_controller.py", line 65, in initialize
    super().initialize()
  File "/conf/apps/controllerx/core/controller.py", line 158, in initialize
    super().initialize()
  File "/conf/apps/controllerx/core/controller.py", line 46, in initialize
    self.controllers_ids = self.get_list(self.args["controller"])
KeyError: 'controller'

2020-02-02 23:44:58.913030 WARNING livingroom_controller: ------------------------------------------------------------

My conf/apps/apps.yaml (is that the right config file?) has this:

livingroom_controller:
  module: controllerx
  class: E1743Controller
  sensor: sensor.ikea_of_sweden_tradfri_on_off_switch_d3e2c8fe_power
  light: light.kitchen_underlight_b01be4fe_level_on_off

I'm not sure how to proceed from here.

[xaviml]

Sorry @mck182, but there has been a breaking change in this release. I haven't said it in here, but you can find it in the README.md from the release. The only difference is that "sensor" need to be changed to "controller" and a new parameter needs to be added "integration". The config in apps.yaml would looks like the following:

livingroom_controller:
  module: controllerx
  class: E1743Controller
  controller: <your device ieee>
  integration: zha
  light: light.kitchen_underlight_b01be4fe_level_on_off

Your device IEEE can be found in Configuration > ZHA and then click on the device you want to use and you will see the device IEEE that looks like XX:XX:XX:XX:XX:XX

[mck182]

A-ha! Thanks, now it runs!

So with that config, I long-press the E1743 buttons but nothing's happening. I double checked the light entity id as well as the controller IEEE and they're correct. How would I go about debugging this further?

[xaviml]

Could you answer the following questions, so I can help you:

• Which light are you using?
• Does turning on/off works?
• Are there any errors in the logs?
• Does your light support brightness?
• Can you change the brightness on Home Assistant?

[mck182]

Which light are you using?

The Ikea countertop one with the Tradfri transformer

Does turning on/off works?

Not through this app, but yes with normal automation

Are there any errors in the logs?

Actually yes, sorry I missed this one:

2020-02-03 00:56:37.525358 WARNING livingroom_controller: ------------------------------------------------------------
2020-02-03 00:56:37.526431 WARNING livingroom_controller: Unexpected error in worker for App livingroom_controller:
2020-02-03 00:56:37.527601 WARNING livingroom_controller: Worker Ags: {'id': '314ccd6db6844791a646d75cae437257', 'name': 'livingroom_controller', 'objectid': 'fdd4a5a98c384ed3b33d5628ed273824', 'type': 'event', 'event': 'zha_event', 'function': <bound method ZHAIntegration.callback of <core.integration.zha.ZHAIntegration object at 0xb5500f88>>, 'data': {}, 'pin_app': True, 'pin_thread': 1, 'kwargs': {'device_ieee': 'cc:cc:cc:ff:fe:02:79:87', '__thread_id': 'MainThread'}}
2020-02-03 00:56:37.529087 WARNING livingroom_controller: ------------------------------------------------------------
2020-02-03 00:56:37.551856 WARNING livingroom_controller: Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/appdaemon/threading.py", line 816, in async_worker
    await funcref(args["event"], data, args["kwargs"])
  File "/conf/apps/controllerx/core/integration/zha.py", line 17, in callback
    action = data["command"]
KeyError: 'command'

Does your light support brightness?
Can you change the brightness on Home Assistant?

Yes and yes.

[xaviml]

Interesting.. could you go to Developer tools > Events
And down the bottom listen for 'zha_event'. Then press the turn on button for example and show me the event that gets printed.

[mck182]

Don't ask me how, but...it's working now. I did bunch of restarts of everything and now I can successfully dim the light up and down.

For posterity, here's the requested log:

{
    "event_type": "zha_event",
    "data": {
        "unique_id": "14:b4:57:ff:fe:c8:e2:d3:1:0x0006",
        "device_ieee": "14:b4:57:ff:fe:c8:e2:d3",
        "endpoint_id": 1,
        "cluster_id": 6,
        "command": "on",
        "args": []
    },
    "origin": "LOCAL",
    "time_fired": "2020-02-03T21:01:10.063616+00:00",
    "context": {
        "id": "b52885d38159495ba396935a14e6501b",
        "parent_id": null,
        "user_id": null
    }
}

I'll still keep an eye on any errors, but works well now as far as I can say!

[xaviml]

Perfect! The error you showed before it was weird because was saying that there was no data on the event. And in the data of the event is where you can find the command and the arguments of the event. I am glad it works then, thanks for the testing!

[jrbenito]

Hi @xaviml ,

First of all THANK YOU for this.

• Which light are you using?
  Ikea GU10 color temp version

• Does turning on/off works?
  Yes

• Are there any errors in the logs?
  2020-02-03 16:06:41 WARNING (MainThread) [zigpy.zcl] Data remains after deserializing ZCL frame

But this is from Zigpy, and seems to happen when bright control reach end of scale (total bright or total dim) and I try one more step.

• Does your light support brightness?
  Yes

• Can you change the brightness on Home Assistant?
  Yes

Now I will try with E1742 and Ikea GU10 that only support brightness and not color temp.

thanks again! Great work

[jrbenito]

Hi @xaviml ,

I grouped some lights with E1810 controller and I still can toggle them on/off but not change color temp nor brightness. I get this on app daemon logs:

020-02-04 01:45:48.493952 WARNING kitchen_controller: ------------------------------------------------------------
2020-02-04 01:45:48.891380 WARNING kitchen_controller: ------------------------------------------------------------
2020-02-04 01:45:48.894126 WARNING kitchen_controller: Unexpected error in worker for App kitchen_controller:
2020-02-04 01:45:48.896690 WARNING kitchen_controller: Worker Ags: {'id': 'cfb4ef7d7c4b4f18bdfc1b22a6284715', 'name': 'kitchen_controller', 'objectid': '95848113c60142c7933055ae0c7978fe', 'type': 'event', 'event': 'zha_event', 'function': <bound method ZHAIntegration.callback of <core.integration.zha.ZHAIntegration object at 0x751f3028>>, 'data': {'unique_id': 'cc:cc:cc:ff:fe:66:c7:b7:1:0x0005', 'device_ieee': 'cc:cc:cc:ff:fe:66:c7:b7', 'endpoint_id': 1, 'cluster_id': 5, 'command': 'press', 'args': [256, 13, 0]}, 'pin_app': True, 'pin_thread': 1, 'kwargs': {'device_ieee': 'cc:cc:cc:ff:fe:66:c7:b7', '__thread_id': 'MainThread'}}
2020-02-04 01:45:48.899347 WARNING kitchen_controller: ------------------------------------------------------------
2020-02-04 01:45:48.903320 WARNING kitchen_controller: Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/appdaemon/threading.py", line 725, in async_worker
    await funcref(args["event"], data, args["kwargs"])
  File "/config/appdaemon/apps/controllerx/core/integration/zha.py", line 22, in callback
    await self.controller.handle_action(action)
  File "/config/appdaemon/apps/controllerx/core/controller.py", line 108, in handle_action
    await action(*args)
  File "/config/appdaemon/apps/controllerx/core/controller.py", line 26, in _action_impl
    continue_call = await self.before_action(method.__name__, *args, **kwargs)
  File "/config/appdaemon/apps/controllerx/core/type/light_controller.py", line 298, in before_action
    light_state = await self.get_entity_state(self.light["name"])
  File "/config/appdaemon/apps/controllerx/core/controller.py", line 131, in get_entity_state
    entity = entities[0]
TypeError: 'NoneType' object is not subscriptable

2020-02-04 01:45:48.906250 WARNING kitchen_controller: ------------------------------------------------------------

If you need anything let me know. I will play aroung more and share what I discover.

[sreknob]

Just did a PR for ICTC-G-1 controller for zha #22

While testing using the beta (2.2.0b3), I did notice a few other things I just want to mention:

1. Light.ON_FULL_BRIGHTNESS is not giving me what I expect. I turns on the light but not at full brightness. Not sure if the attribute is being set properly in light_controller.py or minmax_stepper.py calculation as it appears to just turn on at the first step brightness instead.

2. When using mappings for Light.CLICK_BRIGHTNESS_UP or Light.HOLD_BRIGHTNESS_UP, it only works if the light is already on. Is this by design? At least for the direct binding of the ICTC-G-1 controller to a bulb, any rotation clockwise should turn the bulb on.

Also, AMAZING WORK on this :-D

[xaviml]

Hi @jrbenito

So just to clarify, you can toggle the GU10 (temp version) and also dim and change color temp, but when reaching to the end it does not work? What does it do when it reaches to the end?

And then, regarding the group you created, if you go to HA > developer tools> states and look for the light group, can you turn dim the lights and change the color temp of both of them from there? The error you are getting on the group test is that it does not recognize the entities of the group. How did you create the group? Unfortunately, I think it does not work with light groups, just with plain groups from HA, so:

group:
  - test_lights:
    name: Test lights
    entities:
      - light.light1
      - light.light2

Keep me updated with your findings. Thanks!

[jrbenito]

Hi @xaviml ,

Sorry, I was not clear. With the gu10 (temp version), imagine you are dim (lowest bright possible), then you keep increasing brightness until the maximum. The error I got happens, when bright is on max level, I still try to increase bright. Or, the other way around, if I keep dimming it, after I get to the lowest possible bright, if I press the button to decrease it even more, I get that message on long. Hence, I think there is no problem on your side.

For the group, I created it manually on groups.yml file, I will double check it later at home (I am now at work and with the much snow falling out there I might head home soon or get stuck here). Also will collect info on developer tools. I keep you posted.

thanks for the help and excellent work.

[xaviml]

Thank you @jrbenito for keeping me updated then :)

Let me know if you need any help for the groups.

[jrbenito]

@xaviml ,

Problem solved. There was a capital letter in my group name I did not correctly copied to apps.yaml. Odd that on/off worked without capital letter. Now I can dim/bright, change color temp and on/off with remote.

[jrbenito]

@xaviml

I finally put those E1443 remotes in the mix. They worked well too.

One doubt, in order to have two dimmers for the same set of lamps I did:

hall1_controller:
  module: controllerx
  class: E1743Controller
  controller: xx:xx:xx:11
  integration: zha
  light: group.hallway

hall2_controller:
  module: controllerx
  class: E1743Controller
  controller: xx:xx:xx:22
  integration: zha
  light: group.hallway

Is there another way to do it or this is just right?

thanks

[xaviml]

Hey @jrbenito ,

This is right, but you also have the following options:

hall_controller:
  module: controllerx
  class: E1743Controller
  controller: xx:xx:xx:11, xx:xx:xx:22
  integration: zha
  light: group.hallway

Or

hall_controller:
  module: controllerx
  class: E1743Controller
  controller:
    - xx:xx:xx:11
    - xx:xx:xx:22
  integration: zha
  light: group.hallway

With this two options, you will have just one app intance, instead of 2, but the outcome is the same.

Glad that everything worked :) I will be doing a release in the coming days, I am just working on documentation, the code won't change, except if a bug is found.

[jrbenito]

Much cleaner solution to put both controllers on the same app instance.

How about mixing E1810 with E1743? (I did two separated app instances and worked)

Again thanks for this great job and your continuous support.

[xaviml]

How about mixing E1810 with E1743? (I did two separated app instances and worked)

No, you will need two separate app instances for that. You can merge two controllers in the same app instance if they are the same controller.

Again thanks for this great job and your continuous support.

Thanks for the appreciation :)

[jrbenito]

No, you will need two separate app instances for that. You can merge two controllers in the same app instance if they are the same controller.

I imagine that because differences on controllers. That is cool :)

[xaviml]

I just released v2.2.0 with support to ZHA.