Homebridge-hue: Motion triggers not working

[jdshkolnik]

Hi,

I set up a simple automation rule where a Hue light should turn on if a Hue sensor detects motion. Nothing happens. The Home app shows the sensor as having detected motion but the light stays off. It works if I set it up in the Hue app but not via Home.

[ebaauw]

If Home shows motion detected, the problem is with HomeKit.

HomeKit automation rules run on your HomeKit hub, see __Prerequisites__ in the README. Sometimes the sync over iCloud from your iOS device to your HomeKit hub fails. I find that the following actions typically help: disabling all automations, waiting for these changes to appear on another iOS device, enabling all automations, again waiting for these changes to appear. Unfortunately, there is no way to check the HomeKit configuration on your AppleTV.

[jdshkolnik]

I did try that. I'm also noticing an issue with pre-existing rules. Some are ok, some are delayed, and one isn't working at all.

[ebaauw]

Delayed automation is typically caused by the AppleTV having lost the connection to the accessory. The changed accessory state is then registered by an iOS device, which syncs it through iCloud to the AppleTV. In this case, you'd also see that automations only get executed when an iOS device is awake.

You could try disabling HomeKit on you AppleTV and re-enabling it. Or, more drastically, logout of iCloud on your AppleTV and login again. Or, when everything else fails, reset your entire HomeKit configuration (delete your home and re-pair all accessories - no fun).

The larger your HomeKit configuration gets, the more likely these issues. The HomeKit configuration must fit one CKRecord, or the iCloud synchronisation fails. Typically you'd hit this limit around 150-170 accessories, but your mileage may vary.

[NorthernMan54]

Sorry to hijack this, but with the hue motion sensor does it update HomeKit in real time or is there a delay for polling etc? I have a v1 hub.

On Jun 6, 2017, at 10:21 AM, Erik Baauw [e-mail] wrote:

Delayed automation is typically caused by the AppleTV having lost the connection to the accessory. The changed accessory state is then registered by an iOS device, which syncs it through iCloud to the AppleTV. In this case, you'd also see that automations only get executed when an iOS device is awake.

You could try disabling HomeKit on you AppleTV and re-enabling it. Or, more drastically, logout of iCloud on your AppleTV and login again. Or, when everything else fails, reset your entire HomeKit configuration (delete your home and re-pair all accessories - no fun).

The larger your HomeKit configuration gets, the more likely these issues. The HomeKit configuration must fit one CKRecord, or the iCloud synchronisation fails. Typically you'd hit this limit around 150-170 accessories, but your mileage may vary.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[ebaauw]

@NorthernMan54: Here's what happens when a Hue motion sensor detects motion:

1. The Hue motion sensor, which is configured for ZigBee attribute reporting, sends a ZigBee message to the Hue bridge in real-time;
2. The Hue bridge updates the sensor state and fires any bridge rules in real-time;
3. On the next heartbeat, homebridge-hue detects the changed sensor state and updates the HomeKit characteristic. Technically, a HomeKit-enabled iOS or tvOS device maintains a TCP connection to homebridge, and receives the changed characteristic over this connection in realtime;
4. The HomeKit-enabled iOS or tvOS device syncs the changed characteristic to iCloud;
5. Any HomeKit-enabled iOS or tvOS device not connected to homebridge receives the changed characteristic on their next iCloud sync. I'm not sure if this sync is scheduled or triggered, but typically there is a noticeable delay. Normally, this is needed only for remote devices (not connected to your home network) or for devices out of range of Bluetooth LE accessories.

@jdshkolnik: Depending on whether your HomeKit Hub is connected to homebridge, the HomeKit automation fires on step 3 or step 5. In my experience, issues with HomeKit automation not firing at all are caused by HomeKit automations not having been synced to the HomeKit hub; whereas issues with HomeKit automation firing with a delay, and/or only when another iOS device is awake are caused by the HomeKit hub having lost the connection with the HomeKit accessory (i.c. homebridge).

Edit
You can check with netstat on the machine running homebridge whether the AppleTV actually has a connection to homebridge. The port is listed in the homebridge log:

[6/6/2017, 7:30:31 PM] Homebridge is running on port 51827.

$ netstat | grep .51827
tcp4       0      0  server.stdnkd.51827    iphone-erik.stdn.55201 ESTABLISHED
tcp4       0      0  server.stdnkd.51827    ipad-erik.stdnkd.54089 ESTABLISHED
tcp4       0      0  server.stdnkd.51827    iphone-ing.stdnk.50013 ESTABLISHED
tcp4       0      0  server.stdnkd.51827    atv-living.stdnk.57509 ESTABLISHED

End Edit

The Hue bridge does not support push notifications for changed attributes. Even though this is the second largest discussion thread on the Hue developers forum (after the power-on defaults), it doesn't look like Philips will change this. The dresden elektronik gateway deCONZ does support push notifications, and I plan to use these for homebridge-hue, see issue #106. This would make step 3 real-time as well.

I'm using a 2-second polling interval (heartrate) with homebridge-hue for my v2 Hue bridge. The v1 Hue bridge is way less powerful than the v2 Hue bridge, so you might need to use a slower rate. Based on the Hue motion sensor, I turn on the lights using Hue bridge rules and the Sonos speakers using HomeKit automation. There is a slight but noticeable delay before the speakers turn on, but I guess I learned to live with it.

[NorthernMan54]

tks Erik

I was thinking about buying one, to trigger my front door camera’s, but I’m going to see what other options I have for monitor sensors before pulling the trigger. Thanks for the detailed explanation.

Tks again

On Jun 6, 2017, at 2:24 PM, Erik Baauw [e-mail] wrote:

@NorthernMan54 https://github.com/northernman54: Here's what happens when a Hue motion sensor reports motion:

The Hue motion sensor, which is configured for ZigBee attribute reporting, sends a ZigBee message to the Hue bridge in real-time;
The Hue bridge updates the sensor state and fires any bridge rules in real-time;
On the next heartbeat, homebridge-hue detects the changed sensor state and updates the HomeKit characteristic. Technically, a HomeKit-enabled iOS or tvOS device maintains a TCP connection to homebridge, and receives the changed characteristic over this connection in realtime;
The HomeKit-enabled iOS or tvOS device syncs the changed characteristic to iCloud;
Any HomeKit-enabled iOS or tvOS device not connected to homebridge receives the changed characteristic on their next iCloud sync. I'm not sure if this sync is scheduled or triggered, but typically there is a noticeable delay. Normally, this is needed only for remote devices (not connected to your home network) or for devices out of range of Bluetooth LE accessories.
@jdshkolnik https://github.com/jdshkolnik: Depending on whether your HomeKit Hub is connected to homebridge, the HomeKit automation fires on step 3 or step 5. In my experience, issues with HomeKit automation not firing at all are caused by HomeKit automations not having been synced to the HomeKit hub; whereas issues with HomeKit automation firing with a delay, and/or only when another iOS device is awake are caused by the HomeKit hub having lost the connection with the HomeKit accessory (i.c. homebridge).

The Hue bridge does not support push notifications for changed attributes. Even though this is the second largest discussion thread on the Hue developers forum (after the power-on defaults), it doesn't look like Philips will change this. The dresden elektronik gateway deCONZ does support push notifications, and I plan to use these for homebridge-hue, see issue #106 https://github.com/ebaauw/homebridge-hue/issues/106. This would make step 3 real-time as well.

I'm using a 2-second polling interval (heartrate) with homebridge-hue for my v2 Hue bridge. The v1 Hue bridge is way less powerful than the v2 Hue bridge, so you might need to use a slower rate. Based on the Hue motion sensor, I turn on the lights using Hue bridge rules and the Sonos speakers using HomeKit automation. There is a slight but noticeable delay before the speakers turn on, but I guess I learned to live with it.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/ebaauw/homebridge-hue/issues/118#issuecomment-306574819, or mute the thread https://github.com/notifications/unsubscribe-auth/AS5CmHBCYRKYetSZbPeCTEyC3MHy30_oks5sBZlygaJpZM4NvlC8.

[jdshkolnik]

There is connectivity and I cleaned the bridge. The Hue sensor triggers Hue rules without any real delay yet that same trigger might get a very delayed response or none at all from Home rules.

[jdshkolnik]

I should add that it's only the motion sensors having trouble with the rules. I created a different rule for turning on two Koogeek power outlets when a Wemo switch turns on. It happens right away even though that Wemo switch is enabled via another Homebridge plugin.

[ebaauw]

I've been doing some testing, and there appears to be bug in HomeKit in tvOS 10.2.1. To rule out homebridge-hue, I tested using a motion sensor created in the HomeKit Accessory Simulator. The automation on this sensor fires once, just after being created, but no more. The value of the sensor gets updated alright in the Home app and I do receive a push notification that motion is detected.
When I disable WiFi on my iPad (so it connects to my AppleTV over 4G), I no longer receive the push notifications when motion is detected. Also, the motion sensor status no longer gets updated in Home, even though other services still do.
Setting up the iPad as Home Hub (now again connected to WiFi) and disabling Home on the AppleTV, the automation fires every time, as it should.

It would seem the AppleTV doesn't register the change to the motion sensor state. I've seen a number of people complaining that they didn't receive any motion sensor push notifications over 4G, when I used to receive these. I don't receive them anymore, so I suspect its a recently introduced bug.

I'm afraid I can only offer a workaround: create a CLIPGenericFlag sensor on the Hue bridge, with Hue bridge rules to set/clear the flag based on the Hue motion sensor. Expose the flag as switch to HomeKit, and then base the HomeKit automation on this switch.

[jdshkolnik]

At least you've confirmed it wasn't my imagination. Is there a channel for reporting this bug to Apple?

How do you create a CLIPGenericFlag sensor on the Hue bridge instead of just HomeKit?

On a somewhat related note, is there a way to trigger automation if no motion is detected for a specified interval similar to what we can do in the Hue app?

[ebaauw]

Is there a channel for reporting this bug to Apple?

Not sure. I would like to see confirmation using a true HomeKit enabled motion sensor (like Elgato's Eve Motion), but I only have the Hue motion sensors.

How do you create a CLIPGenericFlag sensor on the Hue bridge

Through the Hue API. I created a little bash script for easier interaction, see https://github.com/ebaauw/ph.sh, so I would issue:

ph_post /sensors '{
    "name": "My Room",
    "type": "CLIPGenericFlag",
    "modelid": "PHCLGF",
    "manufacturername": "Philips",
    "swversion": "1",
    "uniqueid": "/sensors/xxx"
}'

Note that Philips expose it as a "sensor", but it's actually more like a variable on the Hue bridge, which can be read and set through the API and used in Hue bridge rules. See also my notes on home automation.

is there a way to trigger automation if no motion is detected for a specified interval similar to what we can do in the Hue app

The Hue app actually creates a Hue bridge rule with conditions like:

"conditions": [
  {
    "address": "/sensors/xx/state/presence",
    "operator": "eq",
    "value": "false"
  },
  {
    "address": "/sensors/xx/state/lastupdated",
    "operator": "ddx",
    "value": "PT00:00:15"
  }
]

This causes the rule to fire after the motion sensor reports "no motion" for 15 seconds. Note that the interval is hard-coded in the rule. There's no setting for this on the Hue motion sensor (nor on the Hue bridge). You could use a rule with similar conditions to clear the CLIPGenericFlag sensor on the bridge at the end of the interval.

There is no way to express these conditions in HomeKit automation. To overcome that, I included the Duration characteristic in homebridge-hue, even though the Hue motion sensor doesn't support it. Of course, that won't help with the AppleTV not registering the changed Motion state...

[jdshkolnik]

First of all, thank you. Your answers are fantastic. Despite that, I'm still getting through the learning curve. I added the new sensor and it appears in the JSON output for list of sensors yet does not appear in the Hue or Home apps. This is what I get in the CLIP API Debugger:

    "20": {
        "state": {
            "flag": false,
            "lastupdated": "none"
        },
        "config": {
            "on": true,
            "reachable": true
        },
        "name": "Stairway Chandelier Flag",
        "type": "CLIPGenericFlag",
        "modelid": "PHCLGF",
        "manufacturername": "Philips",
        "swversion": "1.0",
        "uniqueid": "Stairway Chandelier Flag 1",
        "recycle": false
    }

Is it not expected to appear in the Hue app? Should I just add conditions through the API for the real sensor to trigger the virtual sensor? How to expose that virtual sensor as a switch to HomeKit?

[ebaauw]

It won't appear in the Hue app. For it to appear in HomeKit, you need to restart homebridge after creating the sensor, see issue #4. Before that, you might want to double-check your config.json, in particular "sensors": true and "excludeSensorTypes", which should _not_ contain "CLIP" (nor "CLIPGenericFlag").

You can create the rules using the CLIP API debugger (POST /rules), with the conditions like in my previous post and actions to set or clear this sensor:

"actions": [
  {
    "address": "/sensors/20/state",
    "method": "PUT",
    "body": {
      "flag": true
    }
  }
]

[jdshkolnik]

@ebaauw If I have my iPhone or iPad open to the Home app when the motion sensor is triggered, the flag turns on and correctly sets off the switch via a Home automation rule. If I don't have the Home app open, the flag turns on but doesn't set off the Home automation rule.

[ebaauw]

Typically this happens when the AppleTV has lost the connection to the accessory (i.c. homebridge). In that case, if one of the iOS devices running Home (or another HomeKit app) connects to the accessory, it receives the new state which is then synced to the AppleTV through iCloud.

__EDIT__ Sorry, I mixed the issues, please see my next post.

[jdshkolnik]

Isn't it related to the bug you found?

[ebaauw]

No. There's (at least) three different issues with HomeKit automations, with different symptoms and different resolutions:

1. __Issue__: The tvOS bug related to the HomeKit Motion Sensor service, when you expose the Hue motion sensor to HomeKit and create a HomeKit automation triggered by the Motion Detected characteristic.
   __Symptoms__: Motion Detected state is updated and notification is received over WiFi, but not over 3G/4G. Automations triggered by Motion Detected never fire; other automations do.
   __Resolution__: None;
   __Workaround__: a) Setup iPad as HomeKit Hub or b) Create a CLIPGenericFlag, which is exposed as Switch so the HomeKit automation is triggered by Power instead of by Motion Detected.
2. __Issue__: Automation config isn't synced to the AppleTV, see https://github.com/ebaauw/homebridge-hue/issues/118#issuecomment-306110826.
   __Symptoms__: Automation isn't fired, ever.
   __Resolution__: Disable/re-enable automations.
3. __Issue__: AppleTV lost connection to the accessory (i.c. homebridge) and doesn't receive the change of the characteristic, so it doesn't fire the automation, see https://github.com/ebaauw/homebridge-hue/issues/118#issuecomment-306501332.
   __Symptoms__: Automation is only fired when HomeKit app on iOS device is active; automation is delayed.
   __Resolution__: Disable/re-enable HomeKit on the AppleTV, or logout from iCloud on the AppleTV and login again.

If I understand correctly, in your case, you've implemented the workaround for 1b) and are now seeing 3).

[jdshkolnik]

I have two AppleTV's and an iPad Pro 12.9 acting as hubs so there is great coverage. I think it might have something to do with the triggering device being a flag exposed via Homebridge. It looks like everything registered via Homebridge (including the Homebridge) show as "Updating" every so often before showing as "No Response" and then returning to normal. Slow polling or something?

[NorthernMan54]

@ebaauw For issue #1, have you seen this hack in HAP-Nodejs? To resolve issues with IOS unregistering motion sensors from events?

https://github.com/jspv/HAP-NodeJS/commit/932ef587bee27cd3788c066d52cfd559ab556b3a

It is discussed here

https://github.com/KhaosT/HAP-NodeJS/issues/418

I have been using it for months with my Motion sensors, and are working fine. These aren't hue sensors though, just ancient X10 Motion Sensors. If I mentioned this before, please excuse me.

[ebaauw]

Thanks @NorthernMan54. Yes, you did mention this in https://github.com/ebaauw/homebridge-hue/issues/125#issuecomment-310899566, and I had a look at it. Not sure I fully understand the issue, see also my reply in the other issue.

I haven't yet tried the hack. Even if it works, I'm not sure how I would publish it as HAP-NodeJS is provided through homebridge instead of being a direct dependency.

[ebaauw]

@jdshkolnik I don't think this is related to homebridge. Similar issues are reported in the Hue developers forum when Hue lights are being exposed by the v2 Hue bridge, see https://developers.meethue.com/content/homekit-no-response-issue.

[jdshkolnik]

That doesn't seem to be the same thing because it's intermittent whereas mine fails to trigger 100% of the time without the app open.

[ebaauw]

@NorthernMan54 I can confirm that this is the same issue as https://github.com/KhaosT/HAP-NodeJS/issues/418. The AppleTV unsubscribes from Motion Detected events. After that, the Motion Detected event is sent to connected iOS devices, but not to the AppleTV. Consequently, remote iOS devices (connected to the AppleTV over 4G) don't receive the updates and automations on the AppleTV do not fire.
I still got a notification on my iPhone over 4G, but probably because the notification received by my iPad was synced over iCloud.

If memory serves, I didn't get any Motion Detected notifications over WiFi on iOS 10.2. I suspect due to the same issue, which Apple fixed in iOS 10.3, but not in tvOS 10.2.

I patched HAP-NodeJS v0.4.26's lib/Accessory.js to ignore the unsubscribe message, as per @jspv's hack. After restarting homebridge, Motion Detected is updated over 4G.

As I said before, I have no clue as to how to publish this hack. Also, I'm a bit concerned by @KhaosT's reservation to merge this hack into HAP-NodeJS. I'd like an additional check that the event actually originates from an AppleTV before ignoring it.

[NorthernMan54]

Ultimately publishing the hack would be Tian's call, as he manages the hap-nodejs repository. For me I added this to my personal version, as well as my Amazon Alexa hack, so all my devices have the fix, and anyone using my version of hap-nodejs ( actually quite a few ). For those watching this, instructions for my version of homebridge with this fix and Alexa support are here

https://github.com/NorthernMan54/homebridge-alexa

I would hope that Apple is aware of the defect and fixes it in one of the upcoming release, if not already.

Sent from my iPad

On Jul 2, 2017, at 7:24 AM, Erik Baauw [e-mail] wrote:

@NorthernMan54 I can confirm that this is the same issue as @NorthernMan54 I can confirm that this is the same issue as KhaosT/HAP-NodeJS#418. The AppleTV unsubscribes from Motion Detected events. After that, the Motion Detected event is sent to connected iOS devices, but not to the AppleTV. Consequently, remote iOS devices (connected to the AppleTV over 4G) don't receive the updates and automations on the AppleTV do not fire.
I still got a notification on my iPhone over 4G, but probably because the notification received by my iPad was synced over iCloud.

If memory serves, I didn't get any Motion Detected notifications over WiFi on iOS 10.2. I suspect due to the same issue, which Apple fixed in iOS 10.3, but not in tvOS 10.2.

I patched HAP-NodeJS v0.4.26's lib/Accessory.js to ignore the unsubscribe message, as per @jsvp's hack. After restarting homebridge, Motion Detected is updated over 4G.

As I said before, I have no clue as to how to publish this hack. Also, I'm a bit concerned by @khaost's reservation to merge this hack into HAP-NodeJS. I'd like an additional check that the event actually originates from an AppleTV.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[ebaauw]

No change in tvOS 10.2.2 as far as I can see.

[CooperCGN]

What's weird, my parents have an Eve Motion installed in the bathroom and it triggers the lights without any iOS device on except their Apple TV4. I tested it yesterday. Now I wonder where is the difference?

[ebaauw]

The most obvious difference is that the Eve Motion uses Bluetooth instead of WiFi.

[CooperCGN]

True, didn't know that a different way of connecting might cause problems.

[nsayer]

" I would like to see confirmation using a true HomeKit enabled motion sensor (like Elgato's Eve Motion)..."

Gotcha covered. I have an Eve Motion and Caseta dimmers and a bunch of AppleTVs. Motion events do register in the home app, and the home app can control the light. But the rule won't turn the light on. Configure an iPad to be an additional home hub and put it within range of everything and the rules work as expected.

Installing the tvOS 11 public beta on our living room AppleTV seems to have fixed it.

[ebaauw]

Installing the tvOS 11 public beta on our living room AppleTV seems to have fixed it.

That is good news, indeed! I don't have a USB-C cable, and only one AppleTV, so haven't been able to test this myself. Shouldn't be long before the production release of tvOS 11 is out, though...

[nsayer]

You can install the beta over-the-air. Go to the public beta site and look for instructions. You may need to sign out of iCloud and back in, but you get a toggle under software update to install beta releases.

[torange]

I previously had some problems with motion sensors triggering automation sequences with iOS/tvOS 10, but as ebaauw has noted, this seemed mainly related to the AppleTV’s poor handling of updated configurations, as the sensor events were still firing in homekit, just failing to trigger automations. iOS/tvOS 11 has been rock solid with zero issues, aside from needing to repair my homebridge with homekit once after updating (renew identity).

I believe this issue can be closed, bearing in mind that iOS 11 should generally be considered a requirement of any stable homekit/homebridge automation setup.

[ebaauw]

Indeed, running on tvOS 11, an automation based on the Hue motion sensor detecting motion is triggered.