Deconz-rest-plugin: General thread on the 0.77 dropping/slow connection issue.

[Mimiix]

Hello there,

Since 0.77, there has been various reports on dropping connections:
https://github.com/dresden-elektronik/deconz-rest-plugin/issues/2863
https://github.com/dresden-elektronik/deconz-rest-plugin/issues/2866

Home-assistant has chosen to revert back to 0.75.
https://github.com/home-assistant/hassio-addons/pull/1375

In regards to that this topic will be the center on solving this issue.

Currently, @manup and other developers are still not sure what causing this. It might has to do with the introduction of APS ACKs(https://github.com/dresden-elektronik/deconz-rest-plugin/pull/2612) . I've been provided with the following Alpha version in which APS ACKs have been disabled. If anyone can test that and share their feedback, please do in this topic!

http://deconz.dresden-elektronik.de/raspbian/alpha/deconz_2.05.77-raspbian-jessie-alpha_armhf.deb

Cheers!
Dennis.

[meremortals70]

So is it being fixed? As when I look at my Deconz it says “commercial version”

Not one response to any of the issues, just a downgrade.

Two months without an update and when it comes it destroys people’s systems.

[Mimiix]

Unfortunately, the Home-assistant Addon is not maintained by DE. Their decision to downgrade is solely their own.

Currently, in Discord devs are trying to figure out whats wrong. Also, the user who created the PR to downgrade the addon in HA is there to fix the "lost connections" issue on that.

Feel free to join the discussion in Discord :)

[Qonstrukt]

I have Deconz running on a separate RPi 2 from my Home Assistant installation on my x86 PC server, and I haven't had any issues. Not sure how useful that info is, just to pitch in that it also might work correctly I guess. 😊

[manup]

Have been running the 2.05.77 beta (not the later alpha with the APS ACKs reverted) in a larger network with 144 devices during the day, with RaspBee II firmware version 0x26570700 which is equal to ConBee II firmware 0x264A0700.

No issues here so far. Note this network doesn't have sleeping end-devices, which have another impact when queuing up many APS requests.

My home network with 58 mixed devices, doesn't show the issues either.

It would be really helpful if anybody experiencing the issues could try the alpha .77 version and provide feedback if that improves the network operation.

http://deconz.dresden-elektronik.de/raspbian/alpha/deconz_2.05.77-raspbian-jessie-alpha_armhf.deb

[MattL0]

Since we are already on beta 77. Why not make it .78 alpha?

[olemr]

Report from openHAB user (2.5.5):
I had to revert back to .75 from .77 as the REST API stopped responding both to my htttPOSTs and the OH2 native binding commands.
After a restart, it worked OK for some 10-12 hours, but come morning, all was broken again.

2020-06-04 19:56:55.404 [ERROR] [.smarthome.model.script.actions.HTTP] - Fatal transport error: java.util.concurrent.TimeoutException: Total timeout 5000 ms elapsed
After downgrading to .75 all has been fine for 48hrs now ...

[easybeat]

Hello there,

Since 0.77, there has been various reports on dropping connections:

2863

2866

Home-assistant has chosen to revert back to 0.75.
home-assistant/hassio-addons#1375

In regards to that this topic will be the center on solving this issue.

Currently, @manup and other developers are still not sure what causing this. It might has to do with the introduction of APS ACKs(#2612) . I've been provided with the following Alpha version in which APS ACKs have been disabled. If anyone can test that and share their feedback, please do in this topic!

http://deconz.dresden-elektronik.de/raspbian/alpha/deconz_2.05.77-raspbian-jessie-alpha_armhf.deb

Cheers!
Dennis.

Hi Dennis,

is #2273 also related to this?

If yes could you please link it and if not could you please have a look into this problem #2273 and maybe give us an update on it?

Thanks

Beat

[Mimiix]

@easybeat It's not connected as the version wasnt out then.

The only thing i've seen is that the database might be corrupt. Go back to basic: if stuff doesnt save, why does that happen? 9/10 it happens because it's some local thing that prevents writing.

In the case of Klucky, He doesn't provide info requested.

Most stuff here isn't just 1 case. If there was a bug that hit all of you, more would have this.

I suggest you to open your own issue as none stand on its own.

[MattL0]

here every time i lauch .78 , Some loose lights. Staying on .77

[Mimiix]

@MattL0 Could you open a new bug report on this?

[Mimiix]

From what i've read and heard, the newest firmware solves the issue. Can anyone elaborate on their issues after the FW update?

[tobalowar]

Hi, since the .78 update my deconz II drop connections every few minutes, making all my sensors being unavailable on home assistant. The previous version .77 works without problems

[vegetate7]

Same for me, on RaspBee (I), Firmware 26350500, rest-plugin v. 2.05.78.
Random disconnects. Show like vertical lines on history chart (not grid!) trougn all deconz managed devices

Begins since upgrade from Firmware 26330500 and rest-plugin v. 2.05.75

[SwoopX]

Since zigbee in general doesn't have any connections (and therefore, neither has deconz), I cannot really follow you.

[vegetate7]

Ok, I'll tel it other way. Some times a day all deconz managed devices became "unavailable". I've collected plugin logs over night, and here is two disconnects:

`
.....
08:34:00:251 don't close database yet, keep open for 900 seconds
08:34:01:522 ZCL attribute report 0x00158D00041D1920 for cluster: 0x0006, ep: 0x0B, frame control: 0x08, mfcode: 0x0000
08:34:06:394 don't close database yet, keep open for 900 seconds
08:34:34:147 don't close database yet, keep open for 900 seconds
08:34:34:147 Remove websocket 172.30.32.1:46514 after error Unknown error
08:34:34:205 no button map for: lumi.relay.c2acn01 ep: 0x01 cl: 0x000A cmd: 0x00 pl[0]: 000
08:34:34:223 no button map for: lumi.relay.c2acn01 ep: 0x01 cl: 0x000A cmd: 0x00 pl[0]: 000
08:35:11:632 don't close database yet, keep open for 900 seconds
08:35:11:638 New websocket 172.30.32.1:55858 (state: 3)
08:35:11:702 Current channel 25
08:35:11:725 Device TTL 4483 s flags: 0x7
08:35:12:100 0xEC1BBDFFFE6C1401 force poll (2)
08:35:12:148 ZCL attribute report 0xEC1BBDFFFE6C1401 for cluster: 0x0006, ep: 0x01, frame control: 0x18, mfcode: 0x0000
08:35:53:237 don't close database yet, keep open for 900 seconds
08:35:53:240 Remove websocket 172.30.32.1:55858 after error Unknown error
08:35:53:256 no button map for: lumi.relay.c2acn01 ep: 0x01 cl: 0x000A cmd: 0x00 pl[0]: 000
08:35:54:211 New websocket 172.30.32.1:55932 (state: 3)
08:35:54:230 Current channel 25
08:35:54:252 Device TTL 4440 s flags: 0x7
08:35:54:743 ZCL attribute report 0xEC1BBDFFFE6C1401 for cluster: 0x0006, ep: 0x01, frame control: 0x18, mfcode: 0x0000
08:35:54:755 0xEC1BBDFFFE6C1401 force poll (2)
08:35:58:204 don't close database yet, keep open for 900 seconds
.....

`

[Mimiix]

@vegetate7 Do you have any logs from the pydeconz component?
https://www.home-assistant.io/integrations/deconz/#debugging-integration

I don't see anything here that is a problem for deCONZ.

[tobalowar]

Hi @Mimiix here i have the logs from pydeconz, as you can see theres a lot of reconnecting:

2020-07-07 10:28:34 DEBUG (MainThread) [pydeconz.websocket] {"attr":{"lastannounced":null,"lastseen":"2020-07-07T14:28:34Z","manufacturername":"dresden elektronik","modelid":"RaspBee","name":"Configuration tool 1","swversion":"0x264a0700","type":"Configuration tool","uniqueid":"00:21:2e:ff:ff:05:ac:17-01"},"e":"changed","id":"1","r":"lights","t":"event","uniqueid":"00:21:2e:ff:ff:05:ac:17-01"}
2020-07-07 10:28:58 DEBUG (MainThread) [pydeconz.websocket] Websocket starting
2020-07-07 10:28:58 DEBUG (MainThread) [pydeconz.websocket] Reconnecting to deCONZ in 15.
2020-07-07 10:29:13 DEBUG (MainThread) [pydeconz.websocket] Websocket starting
2020-07-07 10:29:17 DEBUG (MainThread) [pydeconz.websocket] Websocket running
2020-07-07 10:29:17 DEBUG (MainThread) [pydeconz.websocket] {"config":{"battery":100,"duration":90,"on":true,"reachable":true,"temperature":0},"e":"changed","id":"2","r":"sensors","t":"event","uniqueid":"00:15:8d:00:02:52:aa:f2-01-0406"}
2020-07-07 10:29:18 DEBUG (MainThread) [pydeconz.websocket] {"attr":{"lastannounced":null,"lastseen":"2020-07-07T14:29:18Z","manufacturername":"dresden elektronik","modelid":"RaspBee","name":"Configuration tool 1","swversion":"0x264a0700","type":"Configuration tool","uniqueid":"00:21:2e:ff:ff:05:ac:17-01"},"e":"changed","id":"1","r":"lights","t":"event","uniqueid":"00:21:2e:ff:ff:05:ac:17-01"}
2020-07-07 10:29:22 DEBUG (MainThread) [pydeconz.websocket] {"attr":{"lastannounced":null,"lastseen":"2020-07-07T14:29:22Z","manufacturername":"dresden elektronik","modelid":"RaspBee","name":"Configuration tool 1","swversion":"0x264a0700","type":"Configuration tool","uniqueid":"00:21:2e:ff:ff:05:ac:17-01"},"e":"changed","id":"1","r":"lights","t":"event","uniqueid":"00:21:2e:ff:ff:05:ac:17-01"}
2020-07-07 10:29:39 DEBUG (MainThread) [pydeconz.websocket] {"attr":{"lastannounced":null,"lastseen":"2020-07-07T14:29:39Z","manufacturername":"dresden elektronik","modelid":"RaspBee","name":"Configuration tool 1","swversion":"0x264a0700","type":"Configuration tool","uniqueid":"00:21:2e:ff:ff:05:ac:17-01"},"e":"changed","id":"1","r":"lights","t":"event","uniqueid":"00:21:2e:ff:ff:05:ac:17-01"}
2020-07-07 10:29:43 DEBUG (MainThread) [pydeconz.websocket] {"attr":{"lastannounced":null,"lastseen":"2020-07-07T14:29:43Z","manufacturername":"dresden elektronik","modelid":"RaspBee","name":"Configuration tool 1","swversion":"0x264a0700","type":"Configuration tool","uniqueid":"00:21:2e:ff:ff:05:ac:17-01"},"e":"changed","id":"1","r":"lights","t":"event","uniqueid":"00:21:2e:ff:ff:05:ac:17-01"}
2020-07-07 10:29:49 DEBUG (MainThread) [pydeconz.websocket] {"attr":{"lastannounced":null,"lastseen":"2020-07-07T14:29:49Z","manufacturername":"dresden elektronik","modelid":"RaspBee","name":"Configuration tool 1","swversion":"0x264a0700","type":"Configuration tool","uniqueid":"00:21:2e:ff:ff:05:ac:17-01"},"e":"changed","id":"1","r":"lights","t":"event","uniqueid":"00:21:2e:ff:ff:05:ac:17-01"}
2020-07-07 10:29:53 DEBUG (MainThread) [pydeconz.websocket] {"attr":{"lastannounced":null,"lastseen":"2020-07-07T14:29:53Z","manufacturername":"dresden elektronik","modelid":"RaspBee","name":"Configuration tool 1","swversion":"0x264a0700","type":"Configuration tool","uniqueid":"00:21:2e:ff:ff:05:ac:17-01"},"e":"changed","id":"1","r":"lights","t":"event","uniqueid":"00:21:2e:ff:ff:05:ac:17-01"}
2020-07-07 10:29:57 DEBUG (MainThread) [pydeconz.websocket] {"e":"changed","id":"3","r":"sensors","state":{"lastupdated":"2020-07-07T14:29:57.091","temperature":1782},"t":"event","uniqueid":"00:15:8d:00:02:08:fb:b6-01-0402"}
2020-07-07 10:29:57 DEBUG (MainThread) [pydeconz.websocket] {"e":"changed","id":"4","r":"sensors","state":{"humidity":5751,"lastupdated":"2020-07-07T14:29:57.101"},"t":"event","uniqueid":"00:15:8d:00:02:08:fb:b6-01-0405"}
2020-07-07 10:29:58 DEBUG (MainThread) [pydeconz.websocket] {"attr":{"lastannounced":null,"lastseen":"2020-07-07T14:29:58Z","manufacturername":"dresden elektronik","modelid":"RaspBee","name":"Configuration tool 1","swversion":"0x264a0700","type":"Configuration tool","uniqueid":"00:21:2e:ff:ff:05:ac:17-01"},"e":"changed","id":"1","r":"lights","t":"event","uniqueid":"00:21:2e:ff:ff:05:ac:17-01"}
2020-07-07 10:30:00 DEBUG (MainThread) [pydeconz.websocket] {"attr":{"lastannounced":null,"lastseen":"2020-07-07T14:30:00Z","manufacturername":"dresden elektronik","modelid":"RaspBee","name":"Configuration tool 1","swversion":"0x264a0700","type":"Configuration tool","uniqueid":"00:21:2e:ff:ff:05:ac:17-01"},"e":"changed","id":"1","r":"lights","t":"event","uniqueid":"00:21:2e:ff:ff:05:ac:17-01"}
2020-07-07 10:30:01 DEBUG (MainThread) [pydeconz.websocket] {"e":"changed","id":"2","r":"sensors","state":{"lastupdated":"2020-07-07T14:30:01.254","presence":false},"t":"event","uniqueid":"00:15:8d:00:02:52:aa:f2-01-0406"}
2020-07-07 10:30:25 DEBUG (MainThread) [pydeconz.websocket] Websocket starting
2020-07-07 10:30:25 DEBUG (MainThread) [pydeconz.websocket] Reconnecting to deCONZ in 15.
2020-07-07 10:30:40 DEBUG (MainThread) [pydeconz.websocket] Websocket starting
2020-07-07 10:31:06 DEBUG (MainThread) [pydeconz.websocket] Websocket running
2020-07-07 10:31:06 DEBUG (MainThread) [pydeconz.websocket] {"e":"changed","id":"2","r":"sensors","state":{"lastupdated":"2020-07-07T14:31:06.761","presence":true},"t":"event","uniqueid":"00:15:8d:00:02:52:aa:f2-01-0406"}
2020-07-07 10:31:08 DEBUG (MainThread) [pydeconz.websocket] {"attr":{"lastannounced":null,"lastseen":"2020-07-07T14:31:08Z","manufacturername":"dresden elektronik","modelid":"RaspBee","name":"Configuration tool 1","swversion":"0x264a0700","type":"Configuration tool","uniqueid":"00:21:2e:ff:ff:05:ac:17-01"},"e":"changed","id":"1","r":"lights","t":"event","uniqueid":"00:21:2e:ff:ff:05:ac:17-01"}
2020-07-07 10:31:12 DEBUG (MainThread) [pydeconz.websocket] {"attr":{"lastannounced":null,"lastseen":"2020-07-07T14:31:12Z","manufacturername":"dresden elektronik","modelid":"RaspBee","name":"Configuration tool 1","swversion":"0x264a0700","type":"Configuration tool","uniqueid":"00:21:2e:ff:ff:05:ac:17-01"},"e":"changed","id":"1","r":"lights","t":"event","uniqueid":"00:21:2e:ff:ff:05:ac:17-01"}
2020-07-07 10:31:35 DEBUG (MainThread) [pydeconz.websocket] Websocket starting
2020-07-07 10:31:35 DEBUG (MainThread) [pydeconz.websocket] Reconnecting to deCONZ in 15.

[Smanar]

Like I have said on discord, it s not deconz deconnection by only websocket disconnection.
When you have a device (router) unvailable, try to send it a command to see if it respond.

[tobalowar]

The problem is when in home assistant my devices are unavailable, if i go to Phoscon UI my devices are all available and transmiting without problems (i test my motion sensor and i can see that detect motion when in HA is unavailable). So, for this behavior I assume that the problem is the connection between HA and deconz addon

[Smanar]

So try to check you network config.
You can too connect one more websocket client (there is plugin for browser) and check the connexion.
If the connexion stay alive with the browser plugin and not HA > more chance the problem is from HA client.
If the connexion is killed in same time on the plugin and HA > better explore deconz server.

[tobalowar]

I couldn't connect to websocket, but here theres a log from phoscon where you can see it's removing websockets and creating new ones

[Mimiix]

Websockets are requested by clients (and closed by clients). This is not something deCONZ manages. If everything works in phoscon, you should check with HA.

We do not manage nor maintain the HA Addon

[Mimiix]

As this issue seems to be solved in 0.78, i proceed to close the issue. If there is people having issues on 0.78, please open a own bug report!