Zwave2mqtt: [bug] Error, ERROR: Not enough space in stream buffer

[ksuclipse]

Version

Build/Run method

• [ X] Docker
• [ ] PKG
• [ X] Manually built (git clone - npm install - npm run build ) - manually build docker image

Zwave2Mqtt version: 1.2.3
Openzwave Version: 1.4.1

Describe the bug
after docker is running for a while (at least a day) i start to get many errors of Error, ERROR: Not enough space in stream buffer. This is solved by restarting the container. when this happens nothing talks on the zwave network.

To Reproduce
Steps to reproduce the behavior:
run docker for more than a day

Expected behavior
Not this

Additional context
Running commit: 85b49dcf00c3efdb3825e42d93b265052a9f58a6

[robertsLando]

Could you append log output?

Daniel

On 13 Oct 2019, at 03:44, Errol S. [e-mail] wrote:


Version

Build/Run method

[ X] Docker
PKG
[ X] Manually built (git clone - npm install - npm run build ) - manually build docker image
Zwave2Mqtt version: 1.2.3
Openzwave Version: 1.4.1

Describe the bug
after docker is running for a while (at least a day) i start to get many errors of Error, ERROR: Not enough space in stream buffer. This is solved by restarting the container. when this happens nothing talks on the zwave network.

To Reproduce
Steps to reproduce the behavior:
run docker for more than a day

Expected behavior
Not this

Additional context
Running commit: 85b49dc

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

[Fishwaldo]

This is usually seen on rPI or SBC boards with a Z-Stick. Using a USB powered hub usually resolves it.

(It’s probably related to the recently discovered incompatibilities discovered on rPI 4 boards. )

[robertsLando]

@ksuclipse Can I close this? Did you resolve your issue?

[ksuclipse]

yes not sure what happened but after recent pull its gone

[ksuclipse]

Happened again unfortunately. any thoughts?

OZW_Log.txt

[ksuclipse]

@robertsLando

[robertsLando]

@Fishwaldo could you check this?

[ksuclipse]

any ideas on this? seems to happen after several days of uptime. about to pull the newest commit so i'll keep you posted as well

[Fishwaldo]

See my comment above. It’s a hardware/kernel issue and not anything in OZW we can fix. Try a powered hub. Most reporters say they fix the issue.

[ksuclipse]

I’m directly connected to the nuc running it. Is there a better zwave stick I should be using??

[johntdyer]

I just started getting this as well on Home Assistant 0.102

[Fishwaldo]

As already mentioned, this is in most cases related to power issues with the USB port. Using a powered USB hub solves it most of the time.

[ksuclipse]

In my case at least it is directly connected to the board. Are you saying I
should put a powered hub in between the pc and the zwave receiver??

On Wed, Dec 11, 2019 at 22:38 Justin Hammond [e-mail]
wrote:

As already mentioned this is in most cases related to power issues with
the USB port. Using a powered USB hub solves it most of the time.

—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/OpenZWave/Zwave2Mqtt/issues/141?email_source=notifications&email_token=ABANZTEKFN2BIRSMHMWYIHTQYHE53A5CNFSM4JAFBWL2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGVQXUQ#issuecomment-564857810,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABANZTGTZNHKGGG4HELVFITQYHE53ANCNFSM4JAFBWLQ
.

[Fishwaldo]

Yes.

[robertsLando]

Just curious, is this problem also present when using aeotec z-stick usb gen5?

Daniel

On 12 Dec 2019, at 07:15, Justin Hammond [e-mail] wrote:


Yes.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

[Fishwaldo]

yes - basically internally all sticks are the same... it affects most of them.

[robertsLando]

But why this happens with ozw 1.6 and not 1.4??

Daniel

On 12 Dec 2019, at 08:53, Justin Hammond [e-mail] wrote:


yes - basically internally all sticks are the same... it affects most of them.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

[Fishwaldo]

Its happened since 1.0

It’s a hardware issue…

From: Daniel Lando [e-mail]
Sent: Thursday, 12 December 2019 4:48 PM
To: OpenZWave/Zwave2Mqtt [e-mail]
Cc: Justin Hammond [e-mail]; Comment [e-mail]
Subject: Re: [OpenZWave/Zwave2Mqtt] [bug] Error, ERROR: Not enough space in stream buffer (#141)

But why this happens with ozw 1.6 and not 1.4??

Daniel

On 12 Dec 2019, at 08:53, Justin Hammond <[email protected] [e-mail] > wrote:


yes - basically internally all sticks are the same... it affects most of them.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/OpenZWave/Zwave2Mqtt/issues/141?email_source=notifications&email_token=AAGNURBEMPVGQ56UVJARK5LQYH3CXA5CNFSM4JAFBWL2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGV5HFQ#issuecomment-564908950 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGNURCQ2YOSUHL6UHNVZMDQYH3CXANCNFSM4JAFBWLQ . https://github.com/notifications/beacon/AAGNUREF2IDXCWXPNP6J5A3QYH3CXA5CNFSM4JAFBWL2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGV5HFQ.gif

[nellering]

I just wanted to mention that I am getting this with an aeotec z-stick usb gen5 directly pluged into a Gigabyte 630M motherboard. I will also try a different port and pickup a powered usb hub to see if it fixes the issue.

[johntdyer]

@Fishwaldo

It’s a hardware issue…

Its a hardware issue as in our zWave sticks are broken ? Is there an upstream issue somewhere we can subscribe too for updates ?

[nellering]

I picked up an external powered USB hub, and I am still getting the buffer issue with my Aoetec Gen 5 plugged into the powered hub.

[riemers]

While reading up on things, also issues i can also recall having seen this before a few times. Have not seen it anymore after i started playing more with it. But i have a Nuc with mains power to it. If i see the same stream logs again i'll paste it here too.

[riemers]

OpenZWave Error, ERROR: Not enough space in stream buffer
OpenZWave Error, ERROR: Not enough space in stream buffer
OpenZWave Error, ERROR: Not enough space in stream buffer
OpenZWave Error, ERROR: Not enough space in stream buffer

Was trying to do a fresh install again. Docker logs showed this beforehand. This was without any current zwcfg file.

2019-12-28 16:16:35.605 Detail, Node089, Notification: NodeQueriesComplete
2019-12-28T15:16:35.610Z z2m:Zwave node 88 ready: Unknown: id=0154 - Unknown: type=0100, id=0201 (Unknown)
2019-12-28 16:16:35.612 Detail, Node090, Query Stage Complete (Dynamic)
2019-12-28 16:16:35.612 Detail, Node090, AdvanceQueries queryPending=0 queryRetries=0 queryStage=Configuration live=1
2019-12-28 16:16:35.612 Detail, Node090, QueryStage_Configuration
2019-12-28 16:16:35.612 Detail, Node090, QueryStage_Complete
2019-12-28 16:16:35.612 Warning, CheckCompletedNodeQueries m_allNodesQueried=0 m_awakeNodesQueried=0
2019-12-28 16:16:35.612 Warning, CheckCompletedNodeQueries all=0, deadFound=1 sleepingOnly=0
2019-12-28 16:16:35.612 Detail, Node090, Notification: NodeQueriesComplete
2019-12-28 16:16:35.612 Detail, Node092, Query Stage Complete (Dynamic)
2019-12-28 16:16:35.612 Detail, Node092, AdvanceQueries queryPending=0 queryRetries=0 queryStage=Configuration live=1
2019-12-28 16:16:35.612 Detail, Node092, QueryStage_Configuration
2019-12-28 16:16:35.612 Detail, Node092, QueryStage_Complete
2019-12-28 16:16:35.612 Warning, CheckCompletedNodeQueries m_allNodesQueried=0 m_awakeNodesQueried=0
2019-12-28 16:16:35.612 Warning, CheckCompletedNodeQueries all=0, deadFound=1 sleepingOnly=0
2019-12-28 16:16:35.612 Info, Failed to load device_classes.xml
2019-12-28 16:16:35.612 Info, Check that the config path provided when creating the Manager points to the correct location.
2019-12-28 16:16:35.612 Detail, Node092, Notification: NodeQueriesComplete
2019-12-28T15:16:35.614Z z2m:Zwave node 89 ready: Unknown: id=0154 - Unknown: type=0100, id=0201 (Unknown)
2019-12-28 16:16:35.614 Info, Failed to load device_classes.xml
2019-12-28 16:16:35.614 Info, Check that the config path provided when creating the Manager points to the correct location.
2019-12-28T15:16:35.615Z z2m:Zwave node 90 ready: Unknown: id=0154 - Unknown: type=0100, id=0201 (Unknown)
2019-12-28 16:16:35.615 Detail, Node093, Query Stage Complete (Dynamic)
2019-12-28 16:16:35.615 Detail, Node093, AdvanceQueries queryPending=0 queryRetries=0 queryStage=Configuration live=1
2019-12-28 16:16:35.615 Detail, Node093, QueryStage_Configuration
2019-12-28 16:16:35.615 Detail, Node093, QueryStage_Complete
2019-12-28 16:16:35.615 Warning, CheckCompletedNodeQueries m_allNodesQueried=0 m_awakeNodesQueried=0
2019-12-28 16:16:35.617 Info, Failed to load device_classes.xml
2019-12-28 16:16:35.617 Info, Check that the config path provided when creating the Manager points to the correct location.
2019-12-28T15:16:35.618Z z2m:Zwave node 92 ready: Unknown: id=0154 - Unknown: type=0100, id=0201 (Unknown)
--- multiple http calls ---
2019-12-28 16:24:01.039 Error, ERROR: Not enough space in stream buffer
2019-12-28 16:24:02.039 Error, ERROR: Not enough space in stream buffer
2019-12-28 16:24:02.039 Error, ERROR: Not enough space in stream buffer
2019-12-28 16:24:02.039 Error, ERROR: Not enough space in stream buffer
2019-12-28 16:24:02.039 Error, ERROR: Not enough space in stream buffer
2019-12-28 16:24:02.039 Error, ERROR: Not enough space in stream buffer
2019-12-28 16:24:02.039 Error, ERROR: Not enough space in stream buffer
etc etc etc

[robertsLando]

@Fishwaldo The problem seems still present after using a power hub

[robertsLando]

@Fishwaldo When upgrading to OZW 1.6 I'm getting this problem too, also I'm getting segmentation fault when sending switch multilevel set commands. This was working befor in OZW 1.4

[Fishwaldo]

The stream issue - it’s not a OZW issue. It’s the kernel running out of buffer space to transmit. It seems to be hardware based. I can’t do jack about that unless someone debugs the kernel/hardware.

The multilevel crash is known. The bug was present in OZW 1.4 but it didn’t crash. The fix is very involved and will be what I’m working on soon. (2 attempts at a fix caused other issues).

[hyperbolic2346]

I'm hitting this as well and the WAF is painful. I originally hit this on a pi4 and found this issue. I tried a powered hub as well and no change. I then switched to a different machine, though low-power as well. Could this be related to the buffer defaulting to 2k and we fill that during high load?

https://github.com/OpenZWave/open-zwave/blob/f44387718c9f9509e410dc9c3e8be1aa2b0a5a74/cpp/src/platform/Controller.h#L67

I have no idea the message sizes we pass into this buffer or how many we can generate, but why not bump it and see if this disappears? Maybe add some metrics on the high water of the buffer?

[robertsLando]

@hyperbolic2346 are you using latest version of z2m? docker or manually built? what ozw version are you using?

[hyperbolic2346]

I'm using the latest z2m docker image. It says 1.6.1240 in the control panel. Should I make a new bug since this one was closed?

If I knew how to build a new openzwave and map it into the docker container I would try this change out. An interesting note is that I put z2m on a different physical host that is much more powerful and haven't had the issue. This makes sense because the lower-power machines like the pi4 I was using earlier could get behind on the buffer, but I would expect it to be temporary and so the larger buffer could help there.

[robertsLando]

@hyperbolic2346 I suggest to open this issue on ozw repo, it's not a bug with z2m