Core: Unable to authenticate Hangouts

[troykelly]

Home Assistant release with the issue:
0.86.3, 0.84.5, 0.82.0, 0.81.0, 0.80.3

Last working Home Assistant release (if known):
Never

Operating environment (Hass.io/Docker/Windows/etc.):
Hass.io VMDK image

Component/platform:
Hangouts

Description of problem:
Unable to authenticate.
Using G Suite account, No 2 Factor, Have logged in and used hangouts with the account, Have tried over four weeks leaving days in between each attempt.

Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):

hangouts:
  intents:
    ColorLight:
      sentences:
        - Change the lights to [the color] {color}

Traceback (if applicable):

2018-10-23 10:35:07 INFO (SyncWorker_0) [hangups.auth] Authenticating with refresh token
2018-10-23 10:35:07 INFO (SyncWorker_0) [hangups.auth] Failed to authenticate using refresh token: Refresh token not found
2018-10-23 10:35:07 INFO (SyncWorker_0) [hangups.auth] Authenticating with credentials
2018-10-23 10:35:08 INFO (SyncWorker_0) [hangups.auth] Submitting form on page 'https://accounts.google.com/ServiceLogin'
2018-10-23 10:35:08 INFO (SyncWorker_0) [hangups.auth] Page contains forms: ['gaia_loginform']
2018-10-23 10:35:08 INFO (SyncWorker_0) [hangups.auth] Page contains inputs: [None, None, None, None, None, None, None, None, None, 'profile-information', 'session-state', None, '_utf8', 'bgresponse', 'Email', 'Passwd-hidden', 'next']
2018-10-23 10:35:08 INFO (SyncWorker_0) [hangups.auth] Submitting form on page 'https://accounts.google.com/signin/v1/lookup'
2018-10-23 10:35:08 INFO (SyncWorker_0) [hangups.auth] Page contains forms: ['gaia_loginform']
2018-10-23 10:35:08 INFO (SyncWorker_0) [hangups.auth] Page contains inputs: [None, None, None, None, None, None, None, None, None, None, 'profile-information', 'session-state', '_utf8', 'bgresponse', 'Email-hidden', 'Passwd', 'signIn', None]

Additional information:

[Tim-GitHub]

Same thing for me. I've tried using both a normal Google Gmail account, as well as a GSuite account. No 2 factor authentication on either account. Tried multiple times over a given day, tried waiting several days, and up to a week. Issue still persists. Same errors as the issue reported here.

Tried on 0.80.1, 0.80.2 and 0.80.3, issue continued. Updated to 0.81 and tested, issue persits. Running Hass.io on a Raspberry Pi 3. I'm located in the USA.

"Invalid Login, please try again."

This closed issue also suggests a way to sign in "manually" but I can't finf a way to make it work with hass.io due to the limited options from the docker container: https://github.com/home-assistant/home-assistant/issues/16249

[sads2000]

Same thing for me

[jhuang0]

No success in authenticating on 0.81 for me as well.

[ferdinandovivacqua]

Same thing for me. Version 0.81.0. I think there is some authentication issue in the login procedure.

[Tim-GitHub]

Updated to 0.81.2 and verified issue remains unresolved.

[ferdinandovivacqua]

I'm a new user and I don't know very well the bug procedures.... Shoud we submit any info and or notice to the developers? Is it enough this thread forum?

[saroberts]

Same issue for me too. I just upgraded to version 0.82.0 and it's still broken. I've tried three different Google accounts, some with two-factor by text, some with an app password, and one with "less secure accounts" enabled. In all cases I get the "Invalid Login" error. I've also tried resetting the captcha link at google and still no luck.

Relevant log snippets:

2018-11-10 15:46:49 INFO (SyncWorker_16) [hangups.auth] Authenticating with refresh token
2018-11-10 15:46:49 INFO (SyncWorker_16) [hangups.auth] Failed to authenticate using refresh token: Refresh token not found
2018-11-10 15:46:49 INFO (SyncWorker_16) [hangups.auth] Authenticating with credentials
2018-11-10 15:46:49 INFO (SyncWorker_16) [hangups.auth] Submitting form on page 'https://accounts.google.com/ServiceLogin'
2018-11-10 15:46:49 INFO (SyncWorker_16) [hangups.auth] Page contains forms: ['gaia_loginform']
2018-11-10 15:46:49 INFO (SyncWorker_16) [hangups.auth] Page contains inputs: [None, None, None, None, None, None, None, None, None, 'profile-information', 'session-state', None, '_utf8', 'bgresponse', 'Email', 'Passwd-hidden', 'next']
2018-11-10 15:46:49 INFO (SyncWorker_16) [hangups.auth] Submitting form on page 'https://accounts.google.com/signin/v1/lookup'
2018-11-10 15:46:49 INFO (SyncWorker_16) [hangups.auth] Page contains forms: ['gaia_loginform']
2018-11-10 15:46:49 INFO (SyncWorker_16) [hangups.auth] Page contains inputs: [None, None, None, None, None, None, None, None, None, None, 'profile-information', 'session-state', '_utf8', 'bgresponse', 'Email-hidden', 'Passwd', 'signIn', None]

It seems like that second URL: https://accounts.google.com/signin/v1/lookup is not valid.

[sads2000]

Out of interest, have any of you got Google maps device tracker working?

[saroberts]

@sads2000 good question; I hadn't tried that before. Just tried to set up Google Maps device tracking with my new account that has two-factor authentication on but am using an app password. Upon restarting, get the error:

Error setting up platform google_maps
Traceback (most recent call last):
  File "/usr/src/app/homeassistant/components/device_tracker/__init__.py", line 184, in async_setup_platform
    disc_info)
  File "/usr/local/lib/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/app/homeassistant/components/device_tracker/google_maps.py", line 46, in setup_scanner
    scanner = GoogleMapsScanner(hass, config, see)
  File "/usr/src/app/homeassistant/components/device_tracker/google_maps.py", line 65, in __init__
    hass.config.path(CREDENTIALS_FILE))
  File "/usr/local/lib/python3.6/site-packages/locationsharinglib/locationsharinglib.py", line 386, in __init__
    cookies_file=cookies_file)
  File "/usr/local/lib/python3.6/site-packages/locationsharinglib/locationsharinglib.py", line 210, in __init__
    if cookies_file and self._validate_cookie(cookies_file):
  File "/usr/local/lib/python3.6/site-packages/locationsharinglib/locationsharinglib.py", line 237, in _validate_cookie
    raise InvalidCookies(message)
locationsharinglib.locationsharinglibexceptions.InvalidCookies: The cookies provided do not provide a valid session.Please authenticate normally and save a valid session again

I delete the cookie and restart and get

Could not open pickle file, either file does not exist or no read access.

Error setting up platform google_maps
Traceback (most recent call last):
  File "/usr/src/app/homeassistant/components/device_tracker/__init__.py", line 184, in async_setup_platform
    disc_info)
  File "/usr/local/lib/python3.6/concurrent/futures/thread.py", line 56, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/app/homeassistant/components/device_tracker/google_maps.py", line 46, in setup_scanner
    scanner = GoogleMapsScanner(hass, config, see)
  File "/usr/src/app/homeassistant/components/device_tracker/google_maps.py", line 66, in __init__
    self._update_info()
  File "/usr/src/app/homeassistant/components/device_tracker/google_maps.py", line 78, in _update_info
    for person in self.service.get_all_people():
  File "/usr/local/lib/python3.6/site-packages/locationsharinglib/locationsharinglib.py", line 459, in get_all_people
    people = self.get_shared_people() + [self.get_authenticated_person()]
  File "/usr/local/lib/python3.6/site-packages/locationsharinglib/locationsharinglib.py", line 420, in get_shared_people
    for info in output[0]:
TypeError: 'NoneType' object is not iterable

Restart again and I'm back to the first error. Nothing shows up in my "recently used devices" and I've tried clearing the captcha.

[saroberts]

This seems to be related to:
https://github.com/costastf/locationsharinglib/issues/42
at least for Google Maps. Anybody have an idea if there might be the same issues for Hangouts?

[StyxyDog]

This seems to be related to:
costastf/locationsharinglib#42
at least for Google Maps. Anybody have an idea if there might be the same issues for Hangouts?

Hi I'm using the google maps integration, which was broken, and fixed as per the issue you quoted. I think the issue was because of changes to Google's login form. However the Hangouts integration has never worked for me, possibly the same issue.

[ferdinandovivacqua]

Same problem for me, I'm not able to get hangout integration working. The only workaround for me is using hangups library and use it as shell command. Waiting for a fix of hangout integration

[saroberts]

So using the hangups library directly, without any code modifications, works for you, @ferdinandovivacqua? Because I just installed hangups in a clean/new virtualenv and can't get it to work, either. Just posted an issue over there: https://github.com/tdryer/hangups/issues/438

[saroberts]

Okay, so I made a bit of progress. I was able to use https://gist.github.com/tdryer/b4d4b5944ac22e3b2f3a9dda53e03e87 to manually generate a refresh token using the procedure here https://github.com/tdryer/hangups/issues/350#issuecomment-323553771. However, now that I have that token, I don't know what to do with it? Can I just place that file somewhere in home assistant? I'm running home-assistant in docker, so it would be great if I could just map/mount a file somewhere ...

[ferdinandovivacqua]

Okay, so I made a bit of progress. I was able to use https://gist.github.com/tdryer/b4d4b5944ac22e3b2f3a9dda53e03e87 to manually generate a refresh token using the procedure here tdryer/hangups#350 (comment). However, now that I have that token, I don't know what to do with it? Can I just place that file somewhere in home assistant? I'm running home-assistant in docker, so it would be great if I could just map/mount a file somewhere ...

I've installed the hangups library and followed the indications to get the token (a little bit tricky by the way). Then, I've created an entry in the config.yaml with a shell command like "python3 /home/.... " and I start the script "send message.py" with the right argument. See here https://community.home-assistant.io/t/notifications-google-hangouts/22259/5
However, I don't use hassio or docker, I've installed home assistant in a complete OS (opensuse)

[sads2000]

Great work being so proactive and reporting back to the thread. Sorry I can't be any further help. Watching the posts with interest

[saroberts]

So for somebody who has this working, could you tell me where the refresh_token file is placed on the file system? I now have a token from the manual method, but have no idea where to put it.

[ferdinandovivacqua]

Usually in the cache folder

Il giorno gio 15 nov 2018, 05:44 Scott Roberts [e-mail]
ha scritto:

So for somebody who has this working, could you tell me where the
refresh_token file is placed on the file system? I now have a token from
the manual method, but have no idea where to put it.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/home-assistant/home-assistant/issues/17705#issuecomment-438915785,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AqjlcDN4ZuhgdGZWiktL2uqBaEwVnB2lks5uvPFGgaJpZM4X0c52
.

[Tim-GitHub]

@saroberts: This details the location and fornat - https://github.com/home-assistant/home-assistant/issues/16249

I'm using Hass.io (docker) on my pi3 so I'm not able to navigate to the path specified and modify the files, unfortunately..

[ferdinandovivacqua]

You can (if I remember well it was in /.cache/hangout), but in hassio I wasn't able to make these change permanent, so every reboot of the pi3 you should repeat all the procedure. That's why I reinstalled homeassistant in a complete OS. I use openSUSE

[virtualducttape]

@Tim-GitHub I'm in the same boat, I've jumped through all the hoops to get the refresh_token.txt file from hangups in a standalone install, but since I'm using hass.io, I don't have any way to stick that file where it needs to go.

Is it possible we could get the HA Hangouts component updated to give us the same "manual login" option workaround that's being used to generate this file in Hangups? If the dialog to log in would let me just paste an oauth_code instead of username/password, it seems like that should be sufficient to build the refresh_token.txt and save it where it needs to go on the file system...

[saroberts]

Okay, well thank you all for the help with this. I finally got it figured out. I was largely able to follow the suggestion by @Tim-GitHub at https://github.com/home-assistant/home-assistant/issues/16249#issuecomment-430641355. The only different for me is that I'm using Home Assistant in Docker, so the location to edit is /usr/local/lib/python3.6/site-packages/hangouts/auth.py. I can't wait for this to be fixed correctly in HA.

[Tim-GitHub]

@ferdinandovivacqua I don't appear to have a /.cache/ directory available when SSHing into my Hass.io instance (via the SSH add-on). Any chance you could verify the location you previously used in your Hass.io environment before you switched to OpenSUSE?

@saroberts Glad to hear you got it working in your environment! I know Hass.io is running in a docker environment as well, but apparently I do not have a /usr/local/lib/python3.6/ directory either (also looking via SSH add-on).

[saroberts]

Did you try looking in /usr/lib/..., and not the "local" variant?

@saroberts Glad to hear you got it working in your environment! I know Hass.io is running in a docker environment as well, but apparently I do not have a /usr/local/lib/python3.6/ directory either (also looking via SSH add-on).

[stonith]

I'm not running hass, just the docker image of HA. After running hanups manually and getting the token I put it in /etc/homeassistant/.storage/core.config_entries as per https://github.com/home-assistant/home-assistant/issues/16249#issuecomment-439660129. If that doesn't exist try running find / -name core.config_entries from inside the hass container.

[ferdinandovivacqua]

@ferdinandovivacqua I don't appear to have a /.cache/ directory available when SSHing into my Hass.io instance (via the SSH add-on). Any chance you could verify the location you previously used in your Hass.io environment before you switched to OpenSUSE?

@saroberts Glad to hear you got it working in your environment! I know Hass.io is running in a docker environment as well, but apparently I do not have a /usr/local/lib/python3.6/ directory either (also looking via SSH add-on).

Unfortunately I don't have hassio anymore... I remember it was in a cache directory... can you search for it?

[Tim-GitHub]

To clarify, the Hass.io I am running is the one based on a Docker container within a ResinOS host. The one from here: https://www.home-assistant.io/hassio/installation/

To SSH in I am using the SSH-Addon (https://www.home-assistant.io/addons/ssh/). By design Hass.io's SSH access is severely limited. That said there must be a way to access the core.config_entries or /hangouts/auth.py paths as mentioned above. This documentation seems to indicate a way to SSH into the container to get more control: https://developers.home-assistant.io/docs/en/hassio_debugging.html

@ferdinandovivacqua Is that how you SSH'd into your Hass.io instance? Or did you just use the SSH-Addon directly? Maybe that is the step I am missing.

I attempted to run a find / -name for both core.config_entries and auth.py through the SSH-Addon but with no luck.

[virtualducttape]

I'm running hass.io, and using Configurator, I am able to navigate to /usr/local/lib/python3.6/site-packages/, however there is no /hangouts/ directory there. I've never successfully signed in to Hangouts through the hass.io interface, so perhaps that's why I'm missing this directory?

[ferdinandovivacqua]

As said by virtualducttape, I used the configurator to navigate in the file system. Anyway, as I said above, I didin't follow your procedure, so I didn't change the auth.py. I installed the hangout library in the hassio and then I called by a shell command the python "send_message.py". Unfortunately, I wasn't able to make permanent this change, and after reboot of the raspberry I had to reinstall it. That's why I turned to a complete OS.
Little offtopic: I replaced also the raspberry... I discovered, together with many many people, that SD card of raspberry tend to be damaged... I moved to a complete pc.

[Tim-GitHub]

Okay, using Configurator to browse the file structure instead of the SSH Add-on seems to have gotten me further. Same as @virtualducttape I can now see the site-packages directory in Configurator now, but no Hangouts directory. Tried both local/lib and just lib directories.

@saroberts did you have to create the directory?

[DoctorOctagonapus]

I'm having the same issue setting up Hangouts from within the UI. I'm using a dedicated Google account that I previously used for Maps device tracking with no issue, but it won't let me log in at all. Simply doesn't want to know. 0.86.3 on HassOS 2.8

[rufik]

Same here with v0.87.1.
I had to fetch token manually, then add proper section to core.config_entries and restart HA then.

[teliov]

So this issue seems to stem from the way hangups (the library handling interaction with google hangouts) operates. They previously depended on the google authentication form being designed in a particular way . As an example the use an ID to select the form element within which they can input the supplied email address.

Problem is it seems Google does not used this fixed id any longer. Users of hangups also noticed this.
Good news is the authors of library added (in a new version) some way to handle this case by allowing manual tokens. So it's kind of an automatic way to handle what @rufik did.
I am working on a PR that would allow for providing the token via the interface. I have a crude implementation which gets me in, but it needs some fine-tuning to be production ready.

[cHunter789]

Could someone paste how core.config_entries should be for HA 0.90.1? I have upgraded hangups manually to 0.4.9 and manual login works but I have no idea how to get it work with HA...

[rufik]

I think it's better to wait for https://github.com/home-assistant/home-assistant/pull/22158 to be merged and new HA version released.