Core: 0.112.0 Withings integration not updating sleep mat anymore

[cg089]

Problem

After updating,I had to reauthorize with webhook, which I did .
I deleted the "Profile:" section in the configuration yaml and reauthorized.

Now, there is only sensor values from my scale, but not from my sleep mat anymore (both running in same profile name).

Environment

arch | x86_64
-- | --
chassis | vm
dev | false
docker | true
docker_version | 19.03.8
hassio | true
host_os | HassOS 4.10
installation_type | Home Assistant OS
os_name | Linux
os_version | 5.4.44
python_version | 3.7.7
supervisor | 228
timezone | Europe/Berlin
version | 0.112.0
virtualenv | false

• Home Assistant Core release with the issue:
• Last working Home Assistant Core release (if known):
• Operating environment (OS/Container/Supervised/Core):
• Integration causing this issue:
• Link to integration documentation on our website:

[probot-home-assistant[bot]]

Hey there @vangorra, mind taking a look at this issue as its been labeled with an integration (withings) you are listed as a codeowner for? Thanks!
_{^{(message by CodeOwnersMention)}}

[Bluhme1]

Got the same observation. Think I have seen somewhere that sleep data was removed deliberately because of they couldn’t be trusted

[Bluhme1]

cg089
Do you know whether the “in-bed” sensor is supposed to work with Sleep Mat

[cg089]

The sensor data was perfectly fine until it stopped working.
After updating to 0.112.1 the sensors are still gone.

The "in-bed" binary sensor is there, but seems to be only working when withings is able to call the HA api, e.g. the ports to home assistant are open or nabu-casa is used.

[florianec]

The "in-bed" binary sensor is there, but seems to be only working when withings is able to call the HA api, e.g. the ports to home assistant are open or nabu-casa is used.

I can confirm this. I am using both nabu-casa url and my own subdomain with port cool.subdomain.com:8123 my own subdomain is set-up as external_url.
When i change external_url to nabu-casa url the webhook is working!

So it's maybe the port who is not working with webhook.

@vangorra would it be possible that only the nabu-casa url (when its set-up) is used with webhook?

[vangorra]

Sleep state was removed months ago because the data was entirely unreliable. Withings often made the data available hours after the event occurred.
To address this, webhook support was added. Which ensures you have accurate sensors data. When enabled, withings typically notifies HASS within 5 seconds of a data change.
If you use nabu casa, you don't have to mess with the internal or external urls. The withings component detects if you are using nabu casa. All you should have to check is that the callback URL on your withings developer account matches the documentation.
If you were not using nabu casa, double check the documentation. It's clear on what Withings' requires for a callback server.

[cg089]

From what I can tell, the data was completely reliable and working until last week upon 0.111.x. I don't use nabu casa and don't plan on opening ports.

[florianec]

@vangorra
Sorry that i have to ask again, but i confused things.

Sleep state was removed months ago because the data was entirely unreliable. Withings often made the data available hours after the event occurred.

by this you mean all sleep related things like sleep_deep_duration_seconds, sleep_tosleep_duration_seconds etc.?

To address this, webhook support was added. Which ensures you have accurate sensors data. When enabled, withings typically notifies HASS within 5 seconds of a data change.

For the sleep mat the only thing left now is in_bed with webhook?

If you were not using nabu casa, double check the documentation. It's clear on what Withings' requires for a callback server.

turns out that withings callback url support only 80 and 443 when send notifications through HTTP POST. https://developer.withings.com/oauth2/#tag/glossary
so when external_url is like https://coolname.com:8123 it's never going to work.
Cause webhooks use external_url (if set) with priority and nabu casa as fallback.

[vangorra]

The sleep state entity was removed. It's value was calculated locally and entirely inaccurate. All other sleep entities are aggregated from Withings' sleep summary data. These are available but disabled by default because they do not provide immediate actionable data. These entities can be enabled in the integrations page for your specific profile.
From what I can tell at the time of writing this comment, the data for "in bed" is provided exclusivity by the withings sleep mat. Acquiring this data can only be done with webhooks enabled.
All other data benefits from webhooks being enabled as it means home assistant gets the data without delay sure to polling.
I'm my testing, I was only ever able to get withings to play nice with valid https domains. We're talking certificates from a global certificate authority and domains that are globally resolvable.

[florianec]

Many Thanks! Now it's all clear for me.

These are available but disabled by default because they do not provide immediate actionable data. These entities can be enabled in the integrations page for your specific profile.

For anyone still struggle to find (like me) set filter 'show disabled entities' in withings integration entities page. (top right corner symbol)

[Bluhme1]

Ok great. Good idea with the possibility to enable and disable sensors.
I am still uncertain whether the webhook part works or not.I use Nabo Casa. I cannot get the data in the "in-bed" sensor to change and the others sensor take a while (10 min) to update. I have done exactly as told in the documentation with the callback url and I don't get any errors in the log. There is a webhook entry with the name of Withings on my Nabo Casa cloud page.
Can anybody help

[vangorra]

If sensors are taking 10 minutes to update, that suggests to me that you do not have the webhook feature enabled in the configuration.yml file.

[Bluhme1]

But I have use_webhook: true in the config yaml

[Bluhme1]

I trie using the URL that Home Assistant created in the web hook section and get the message "405 Method not allowed"

[vangorra]

That is an acceptable response from the home assistant web hook. It only supports POST and HEAD methods.
Since you have the config value set, that should do the trick. On startup (and every 2 hours), the integration will re -etup the webhook subscriptions with withings.

Additionally, when web hooks are enabled, polling occurs every 2 hours instead of every 10 minutes. Since you are seeing data updates more frequent than 2 hours, this suggests to me that the web hooks are working. Withings can be delayed in their delivery of data.
For the sleep mat, my experience has been roughly 10 seconds before the webhook is notified of the change. If you are not seeing that, it's possible the mat is not detecting you. Placement of the mat can be tricky.

[Bluhme1]

Now everything works. The sleep mat wasn't correctly placed - as you suggested.

Thank you for a beautiful integration :-)

[Bluhme1]

Hi Robert
You are probably the most knowlegeable person concerning the Withing API. I have a Sleep Mat and in the app Health Mate and on the website there is information concerning sleep breathing, sleep heart rate and sleep score among other things. Your integration make sensors for the above mentioned data but they are not populated with data. State is "unknown".
Do you know why those data are not exported from the API? I have tried asking the Withings API team several times but the don't answer back. It seems thar the are not very interested in making data available for third party users.
It is not a very serious problem as the integration as a whole works beautifully but it would be nice to know.
Thanks in advance

[JamieP83]

hey,
sorry, i cant seem to get the "in_bed" sensor working.

it just stays on "off", even when ive just been to bed for 8+hours
withings:
client_id: !secret WithingsClientID
client_secret: !secret WithingsClientSecret
use_webhook: true

some of the other sleep values seem to be working ok