Core: New HTML5 notifications fail

[mattsch]

Make sure you are running the latest version of Home Assistant before reporting an issue.

You should only file an issue if you found a bug. Feature and enhancement requests should go in the Feature Requests section of our community forum:

Home Assistant release (hass --version):
0.27.0

Python release (python3 --version):
3.4.2

Component/platform:
notify.html5/raspi3

Description of problem:
Notifications fail to send

Expected:
Browser notifications via Chrome (desktop tested)

Problem-relevant configuration.yaml entries and steps to reproduce:

  - platform: html5
    name: HTML5
    gcm_api_key: MASKED
    gcm_sender_id: MASKED

1. Enable component
2. Send notification (used example data from docs)
3. Traceback

Traceback (if applicable):

Aug 28 14:06:26 bacchus hass[13463]: INFO:homeassistant.core:Bus:Handling <Event call_service[L]: service_call_id=1978536272-9, service_data=message=Anne has arrived home, data=actions=[{'action': 'open', 'title': 'Open Home Assistant', 'icon': '/static/icons/favicon-192x192.png'}, {'action': 'open_door', 'title': 'Open door'}], domain=notify, service=html5>
Aug 28 14:06:26 bacchus hass[13463]: ERROR:homeassistant.core:BusHandler:Exception doing job
Aug 28 14:06:26 bacchus hass[13463]: Traceback (most recent call last):
Aug 28 14:06:26 bacchus hass[13463]: File "/usr/local/lib/python3.4/dist-packages/homeassistant/core.py", line 852, in job_handler
Aug 28 14:06:26 bacchus hass[13463]: func(*args)
Aug 28 14:06:26 bacchus hass[13463]: File "/usr/local/lib/python3.4/dist-packages/homeassistant/core.py", line 717, in _execute_service
Aug 28 14:06:26 bacchus hass[13463]: service(call)
Aug 28 14:06:26 bacchus hass[13463]: File "/usr/local/lib/python3.4/dist-packages/homeassistant/core.py", line 572, in __call__
Aug 28 14:06:26 bacchus hass[13463]: self.func(call)
Aug 28 14:06:26 bacchus hass[13463]: File "/usr/local/lib/python3.4/dist-packages/homeassistant/components/notify/__init__.py", line 106, in notify_message
Aug 28 14:06:26 bacchus hass[13463]: File "/home/pi/.homeassistant/deps/pywebpush/__init__.py", line 170, in send
Aug 28 14:06:26 bacchus hass[13463]: encoded = self.encode(data)
Aug 28 14:06:26 bacchus hass[13463]: File "/home/pi/.homeassistant/deps/pywebpush/__init__.py", line 146, in encode
Aug 28 14:06:26 bacchus hass[13463]: authSecret=self.auth_key)
Aug 28 14:06:26 bacchus hass[13463]: File "/home/pi/.homeassistant/deps/http_ece/__init__.py", line 141, in encrypt
Aug 28 14:06:26 bacchus hass[13463]: authSecret=authSecret, padSize=padSize)
Aug 28 14:06:26 bacchus hass[13463]: File "/home/pi/.homeassistant/deps/http_ece/__init__.py", line 62, in deriveKey
Aug 28 14:06:26 bacchus hass[13463]: backend=default_backend()
Aug 28 14:06:26 bacchus hass[13463]: File "/home/pi/.homeassistant/deps/cryptography/hazmat/backends/__init__.py", line 35, in default_backend
Aug 28 14:06:26 bacchus hass[13463]: _default_backend = MultiBackend(_available_backends())
Aug 28 14:06:26 bacchus hass[13463]: File "/home/pi/.homeassistant/deps/cryptography/hazmat/backends/multibackend.py", line 33, in __init__
Aug 28 14:06:26 bacchus hass[13463]: "Multibackend cannot be initialized with no backends. If you "
Aug 28 14:06:26 bacchus hass[13463]: ValueError: Multibackend cannot be initialized with no backends. If you are seeing this error when trying to use default_backend() please try uninstalling and reinstalling cryptography.

Additional info:

It's worth noting that extra dev libraries are needed to get the component to install, more info here

[happyleavesaoc]

I got this when I used the wrong sender ID. It is your "project ID".

[mattsch]

I used the 'Project Number' as specified by the linked documentation. I did try the 'Project ID' as well just now but got the same behavior.

[sgauche]

I am seeing the same behavior.

bash 16-08-29 17:36:40 homeassistant.core: BusHandler:Exception doing job Traceback (most recent call last): File "/srv/hass/hass_venv/lib/python3.4/site-packages/homeassistant/core.py", line 852, in job_handler func(*args) File "/srv/hass/hass_venv/lib/python3.4/site-packages/homeassistant/core.py", line 717, in _execute_service service(call) File "/srv/hass/hass_venv/lib/python3.4/site-packages/homeassistant/core.py", line 572, in __call__ self.func(call) File "/srv/hass/hass_venv/lib/python3.4/site-packages/homeassistant/components/notify/__init__.py", line 106, in notify_message data=data) File "/srv/hass/hass_venv/lib/python3.4/site-packages/homeassistant/components/notify/html5.py", line 379, in send_message json.dumps(payload), gcm_key=self._gcm_key, ttl='86400') File "/home/hass/.homeassistant/deps/pywebpush/__init__.py", line 170, in send encoded = self.encode(data) File "/home/hass/.homeassistant/deps/pywebpush/__init__.py", line 146, in encode authSecret=self.auth_key) File "/home/hass/.homeassistant/deps/http_ece/__init__.py", line 141, in encrypt authSecret=authSecret, padSize=padSize) File "/home/hass/.homeassistant/deps/http_ece/__init__.py", line 62, in deriveKey backend=default_backend() File "/home/hass/.homeassistant/deps/cryptography/hazmat/backends/__init__.py", line 35, in default_backend _default_backend = MultiBackend(_available_backends()) File "/home/hass/.homeassistant/deps/cryptography/hazmat/backends/multibackend.py", line 33, in __init__ "Multibackend cannot be initialized with no backends. If you " ValueError: Multibackend cannot be initialized with no backends. If you are seeing this error when trying to use default_backend() please try uninstalling and reinstalling cryptography.```

[rbray89]

I see this issue as well. Tried re-installing cryptography to no avail.

[robbiet480]

Can you try installing:

• libffi-dev
• libpython-dev
• libssl-dev

Should be just sudo apt-get install libpython-dev libffi-dev libssl-dev.

It was mentioned in #3046.

[sgauche]

@robbiet480

pi@raspberrypi:~ $ sudo apt-get install libffi-dev libssl-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
libffi-dev is already the newest version.
libssl-dev is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 6 not upgraded.

[robbiet480]

Can y'all please run pip uninstall pywebpush && pip install "https://github.com/web-push-libs/pywebpush/archive/e743dc92558fc62178d255c0018920d74fa778ed.zip#pywebpush==0.5.0". If you run in a virtualenv then activate it first.

I believe that might fix things. It seems like pywebpush isn't being correctly installed

Let me know what you find. If this works for everyone i'll have to figure out what's wrong internally that causes the failed install and push a fix.

[robbiet480]

Oh before running the command I gave please make sure all 3 of the packages I mentioned are installed (libpython-dev libffi-dev libssl-dev).

[robbiet480]

Let me summarize:

1. Shutdown Home Assistant
2. If you use a virtual environment (virtualenv/venv), activate it.
3. sudo apt-get install libpython-dev libffi-dev libssl-dev
4. pip uninstall pywebpush
5. pip install "https://github.com/web-push-libs/pywebpush/archive/e743dc92558fc62178d255c0018920d74fa778ed.zip#pywebpush==0.5.0"
6. Start up Home Assistant
7. Attempt to send a notification over HTML5.

[bazis]

I have the same errors, when trying to send push messages. HASS 0.27.1, windows 10.

[robbiet480]

@bazis Please try the steps I listed in the comment above yours.

[bazis]

I don't know how to install deb packages in windows ;) I'll try only this
pip uninstall pywebpush
pip install "https://github.com/web-push-libs/pywebpush/archive/e743dc92558fc62178d255c0018920d74fa778ed.zip#pywebpush==0.5.0"

[bazis]

Somehow pywebpush was not installed, so I run
pip3 install pywebpush and notification sucessfully sended without errors, Thanks.

[robbiet480]

@bazis Ah yeah, for Windows you just need to run the pip commands as per the Installation instructions.

[robbiet480]

Perfect!

[bazis]

Almost perfect...
Try to turn on push notifications from phone with another google account. Get this error
homeassistant.components.notify.html5: Saving config file failed: b'Ko_tZNnXXXXXXXXXXXCA==' is not JSON serializable
And after that html5_push_registrations.conf became empty. But notifications to first phone still arrives.

[robbiet480]

I've seen that bug once before but we were never able to track it down. You just need to re-register your device and everything should be fine going forward.

[happyleavesaoc]

FWIW, I also experienced that serialization issue when toggling push notifications.

[bazis]

Restart HA, register fist phone with my google account, all ok, then second with different - got the same error every time I toggle enable push notification in HASS UI.
html5_push_registrations.conf empty again.

UPD:
Restart HA, register second phone, then first - no errors.

[rbray89]

I'm seeing the same issue with regards to multiple notifications. I think it has to do with HASS restarts messing up the registers/deregisters.

[robbiet480]

Okay, i'll take a look at the registration problem in depth shortly.

In the meantime, is everyone able to receive notifications now that you have manually installed pywebpush?

[cperezpenichet]

Yes, I can confirm I can receive notifications after manually installing pywebpush.

[bazis]

Mine works too.

[robbiet480]

@rbray89 @bazis @happyleavesaoc I'm unable to reproduce the registrations issue. Can you make a quick edit and try again to see if this fixes the issue?

1. Open home-assistant/homeassistant/components/notify/html5.py.
2. On line 147 remove , indent=4, sort_keys=True.

So the old line was:

fdesc.write(json.dumps(config, indent=4, sort_keys=True))

Now it should be:

fdesc.write(json.dumps(config))

Let me know if that works!

[robbiet480]

As for the pywebpush issue, I believe that it may fail to install if you don't have the packages installed in advance. Unless someone else can come up with another reasonable possibility, I'm going to fix this issue by updating the documentation.

[rbray89]

@robbiet480 I'll try this, if not, I'll try and find the exact steps to reproduce. I know there was a restart of HA, as well as a deregistration involved as well.

[ktpx]

Fails unless its installed with sudo. Completed then, but registration never works (no errors in log either).

[robbiet480]

@ktpx What fails without sudo? sudo apt-get install libpython-dev libffi-dev libssl-dev or pip install "https://github.com/web-push-libs/pywebpush/archive/e743dc92558fc62178d255c0018920d74fa778ed.zip#pywebpush==0.5.0"?

[rpitera]

@robbiet480 For me it was the pip install. With sudo it ran and HA appears to be starting up now - fingers crossed....

[rpitera]

Hope I got the gcm_sender_id right; it wasn't clear where this was. I used the number at the end of the Project ID; in my case push-notifications-XXXXX - I used the number represented by the X's here.

[rpitera]

Hmmm. When I tried to set the slider on my desktop Chrome, I got the message "Notification Registration Failed".
But on my Droid Turbo, it appears to have worked. Created the conf file anyway...

[sgauche]

@rpitera open the Google developers console and open your project. Click the 3 little dots at the top right and click Project settings. Your project number should be there.

[rpitera]

@sgauche so gcm-sender-id = project number? That must be why my test message didn't work.

[sgauche]

Correct

[rpitera]

@sgauche Thanks Scott; two more things (because I made the change and the restarted but still am not getting any messages on the droid)

1. In the docs, the gcm-sender-key and gcm-sender-id examples are enclosed in single quotes; was that just for display or do I need to enclose those values in single quotes?
2. After making the changes in the configs mentioned, to I need to 'resubscribe' on my Droid by turning off and then turning back on the Push Notification slider?

[robbiet480]

@rpitera yes and yes

[rpitera]

Thanks @robbiet480 - making the changes and will report back in a moment...

[rpitera]

1. Have an html_push_registrations.conf that's empty now
2. Got this when I tried to send the message
   16-09-02 21:16:27 homeassistant.core: BusHandler:Exception doing job Traceback (most recent call last): File "/srv/hass/hass_venv/lib/python3.4/site-packages/homeassistant/core.py", line 852, in job_handler func(*args) File "/srv/hass/hass_venv/lib/python3.4/site-packages/homeassistant/core.py", line 717, in _execute_service service(call) File "/srv/hass/hass_venv/lib/python3.4/site-packages/homeassistant/core.py", line 572, in __call__ self.func(call) File "/srv/hass/hass_venv/lib/python3.4/site-packages/homeassistant/components/notify/__init__.py", line 106, in notify_message data=data) File "/srv/hass/hass_venv/lib/python3.4/site-packages/homeassistant/components/notify/html5.py", line 379, in send_message json.dumps(payload), gcm_key=self._gcm_key, ttl='86400') File "/home/hass/deps/pywebpush/__init__.py", line 170, in send encoded = self.encode(data) File "/home/hass/deps/pywebpush/__init__.py", line 146, in encode authSecret=self.auth_key) File "/home/hass/deps/http_ece/__init__.py", line 141, in encrypt authSecret=authSecret, padSize=padSize) File "/home/hass/deps/http_ece/__init__.py", line 62, in deriveKey backend=default_backend() File "/home/hass/deps/cryptography/hazmat/backends/__init__.py", line 35, in default_backend _default_backend = MultiBackend(_available_backends()) File "/home/hass/deps/cryptography/hazmat/backends/multibackend.py", line 33, in __init__ "Multibackend cannot be initialized with no backends. If you " ValueError: Multibackend cannot be initialized with no backends. If you are seeing this error when trying to use default_backend() please try uninstalling and reinstalling cryptography.

[robbiet480]

@rpitera And you manually installed pywebpush again?

[rpitera]

Going to step away from this for a few and grab some dinner... I think I may be suffering from "HA Malnutrition Syndrome" - you know, when you miss dinner because you're this close to getting something to work... : )

BRB

[rpitera]

And you manually installed pywebpush again?
No, didn't realize I had to do that - hang on.....

[robbiet480]

@rpitera Follow these instructions please

[rpitera]

I've followed those instructions once already; are you saying I should do them all again?

[robbiet480]

@rpitera If you wouldn't mind, yes :)

[rpitera]

I don't mind doing anything you tell me, you're helping me out here! Thanks!

Stand by....

[rpitera]

Bummer. I was able to unsubscribe and resub on the droid and that got me a valid conf file.
But I'm getting the same error when I try to send a test message.

And this time running the instructions I used sudo for both of these lines (which seemed to work better):
pip uninstall pywebpush
pip install "https://github.com/web-push-libs/pywebpush/archive/e743dc92558fc62178d255c0018920d74fa778ed.zip#pywebpush==0.5.0"

[rpitera]

Yeah, just double checked and it's the same as the error I posted previously.

I'm going to grab some food and come back to this in a bit. I'll check here as soon as I'm done. I want to thank you and Scott for taking the time to help me!

[rpitera]

One more thing: My environment is RasPi 3, AIO installed, HA at latest build (27.1)
I am using a venv and did activate it before running through the steps mentioned.

[robbiet480]

@rpitera I can keep helping you on Gitter.

I'm calling the original issue (the multibackend error) fixed for now and closing this. If people continue having issues with the registrations file we can open a new issue.

[rpitera]

I don't know how to use Gitter. I guess I'll just give up on this and see if someone in the forums can help me. I have run through the same steps three times now and get the same multibackend error.

[sgauche]

@rpitera To chat on gitter, go here and log in with your GitHub credentials. https://gitter.im/home-assistant/home-assistant

[rpitera]

I appreciate it but I know how to get there, I don't know how to operate in it. Too many simultaneous conversations going on and I can't follow anything. I've posted in the forums in hopes that someone can help me there. As I am still getting the exact same message as the issue, for me it's not closed.

[ktpx]

@robbiet480 apt-get generally needs sudo. the pip install, at least for me, I had to use sudo.

Edit: everything seemingly now works (registration goes through, the config is created). Great! However, I never receive notifications, anywhere. No errors are thrown, the service call is logged like it is successful.

[norrisp90]

I found a fix for the MultiBackEnd error on RPI 3, all-in-one-installer. If you modify html5.py to include this patch function: https://github.com/pyca/cryptography/issues/2039

I added it after the save config function and called it after the registrations dictionary check. Now notifications work.

[robbiet480]

@norrisp90 I'm working on HTML5 right now and will add this patch if it looks good. Thanks for the tip!

[norrisp90]

Here's the patch code.

def patch_crypto_be_discovery():

    """
    Monkey patches cryptography's backend detection.
    Objective: support pyinstaller freezing.
    """

    from cryptography.hazmat import backends

    try:
        from cryptography.hazmat.backends.commoncrypto.backend import backend as be_cc
    except ImportError:
        be_cc = None

    try:
        from cryptography.hazmat.backends.openssl.backend import backend as be_ossl
    except ImportError:
        be_ossl = None

    backends._available_backends_list = [
        be for be in (be_cc, be_ossl) if be is not None
    ]

I'm call it:

if registrations is None:
        return None

    patch_crypto_be_discovery()

    hass.wsgi.register_view(
        HTML5PushRegistrationView(hass, registrations, json_path))
    hass.wsgi.register_view(HTML5PushCallbackView(hass, registrations))

[sgauche]

@robbiet480 having to go through setting up hass again, I had to change the order in which I did things from your list above. Just swap step 2 and 3.

1. Shutdown Home Assistant
2. sudo apt-get install libpython-dev libffi-dev libssl-dev
3. If you use a virtual environment (virtualenv/venv), activate it.
4. pip uninstall pywebpush
5. pip install "https://github.com/web-push-libs/pywebpush/archive/e743dc92558fc62178d255c0018920d74fa778ed.zip#pywebpush==0.5.0"
6. Start up Home Assistant
7. Attempt to send a notification over HTML5.

[nivekmai]

Just updated to 0.30.2 and needed to put @norrisp90's patch back in, has that not made it to release yet?

[robbiet480]

@nivekmai It did not, because from what I read there was a very similar patch put in upstream in pycryptography. I'll take a look again at it later and see if we still need it. Or maybe i'm just behind on updating to the latest pywebpush.

[sgauche]

I still had to do the above on 0.30.1 to get push notifications working.

[Molodax]

Hi,

please note that the suggested fix did not work in my case regarding the issue Saving config file failed: b'BKFXXXXXXXXXXXXX_XX-c=' is not JSON serializable, see the link .

[joseph-holland]

Hi Guys,
Just to add a note to this it seems that it's not possible to enable/disable the html5 push notifications when the webserver is looking for a username/password (auth_basic in nginx, apache, etc.).

I was trying to debug this issue and used the steps above to complete the manual install of pywebpush but still could not get the push notifications working. When I disabled password auth on the webserver (but still kept the api_password on), I was able to enable the notifications for multiple devices.

Even when the password auth is enabled again the notifications still work, it just seems that it's interfering with the enabling/disabling of it.

Just thought I'd put a note here about this just in-case some others had the same problem as myself.

Joe.

[Molodax]

Hi @joseph-holland
Could you please explain how to do this?

I disabled password auth on the webserver (but still kept the api_password on)

I have an issue here, so hope it could help.
Thank you in advance.

[Chris-V]

On Arch the -dev packages don't exist as headers are usually included in the base package. Note that I also had to install pkg-config (sudo pacman -Sy pkg-config) or pywebpush#0.5.0 install would fail on include <ffi.h>.

[dony71]

I have installed latest pywebpush 0.6.1 and homeassistant 0.33.2
When starting homeassistant push notification slider is ON, however when I slider it OFF and ON again,
I got the message "Notification Registration Failed".
I use Project Number as gcm_sender_id and Web API Key as gcm_api_key
There is no any html5_push_registrations.conf generated in .homeassistant folder
Is there something I'm missing?

[sonic2wb]

Just had this issue last night Followed these instructions to the letter and now everything works. HA will auto download and install pywebpush 6.0. (at least it did for me) after it started back up.
everything works like a charm. Thank You to the developers.

Shutdown Home Assistant
sudo apt-get install libpython-dev libffi-dev libssl-dev
Start up Home Assistant
Attempt to send a notification over HTML5.

[macleajb]

Hi Folks,

Just wanted to comment that the issue fixed by @joseph-holland was also the problem on my end. Allowing the request to manifest.json through without authentication made my 'Push Notifications' button work in Chrome.

Thanks,
JES

[Darklyte]

@macleajb @joseph-holland

Could either of you explain what you did exactly? as a linux newbie I'm pretty confused. It seems you both fixed the only issue I am having, which is chrome not receiving the notifications while firefox does.

[macleajb]

To @Darlyte.

If you need to supply a userid and a password to login to your Home Assistant from remote sites, then you may hit this problem. When I looked in the console (ctrl-shift-C) of my browser (Chrome) where it would shovel out the errors, I noticed that https:///manifest.json was returning error 401. What I did to work around this is to allow requests for that file to be made with out authentication by modifying my Apache with something similar to:

<Location /> AuthType Basic AuthName "Protected site" Require expr %{REQUEST_URI} =~ /manifest.json/ <If "! %{REQUEST_URI} =~ /manifest.json/"> Require valid-user </If> </Location>

[Darklyte]

@macleajb

Thank you for the quick reply. I thought that might be the problem I was having, but I do not need to put in a userid and password to connect.

I think it may be another issue, I'm not sure if it is something I've changed in my fiddling, but I can't even connect with chrome anymore. My console gives errors

Failed to load resource: net::ERR_CONNECTION_TIMED_OUT

Even in Firefox, it seems to have issues and I'm sure it has been having these for a while. As soon as I log in it says connection lost and I get this error in the console (repeatedly)

Firefox can’t establish a connection to the server at wss://xxxxx.duckdns.org/api/websocket. core-ad1ebcd0614c98a390d982087a7ca75c.js:3:28874

'uncaught exception: 1'

[aelg305]

I was able to install on Windows 10, my steps:
-installed python
-install HA
-then CMD:
python pip install "https://github.com/web-push-libs/pywebpush/archive/e743dc92558fc62178d255c0018920d74fa778ed.zip#pywebpush==0.5.0"
python -m pip install --upgrade pywebpush

that would install pywebpush 0.6.1.... latest available to date.
Windows 10 Home Assistant pywebpush with: libffi-dev, libpython-dev, and libssl-dev