Core: Google Calendar integration not creating google_calendars.yaml (sudden error, was working before)

[ASNNetworks]

The problem
As of 9 hours ago Google Calendar integration suddenly stopped working. First I got the 505 error. So I deleted the .google_token, restarted HA and authenticated HA again with Google Calendar. Then I got the 404 error. So I figured, well let's remove the google_calendars.yaml as well and let HA create a new one. Deleted that and restarted HA. No yaml file was created. Then I deleted the .google_token yet again do start the proces again. Restarted HA, reauthenticated, (it even says the yaml is created at the notifications) but no yaml file is created. I even removed the client id and secret and created an entire new project. Filled in everything (also re-authenticated succefully again), still no yaml file. I reverted to the previous client id, re-authenticated yet again (got a notification saying the yaml file is created) and yet again: no google_calendars.yaml file. I have no issues using Google Calendars online (or using the apps) and have the API enabled on both projects.

Environment
Hostname | homeassistant
-- | --
System | HassOS 4.11
Deployment | production
Supervisor: 228
Running 0.112.4

• Home Assistant Core release with the issue:
• Last working Home Assistant Core release (if known):
• Operating environment (OS/Container/Supervised/Core):
• Integration causing this issue:
• Link to integration documentation on our website:

Problem-relevant configuration.yaml
google:
client_id: !secret google_calendar_client_id
client_secret: !secret google_calendar_client_secret

Traceback/Error logs
Logger: homeassistant.core
Source: components/google/__init__.py:349
First occurred: 9:14:16 (1 occurrences)
Last logged: 9:14:16

Error executing service: Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/core.py", line 1276, in catch_exceptions
await coro_or_task
File "/usr/src/homeassistant/homeassistant/core.py", line 1299, in _execute_service
await self._hass.async_add_executor_job(handler.func, service_call)
File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(self.args, *self.kwargs)
File "/usr/src/homeassistant/homeassistant/components/google/__init__.py", line 257, in _scan_for_calendars
service = calendar_service.get()
File "/usr/src/homeassistant/homeassistant/components/google/__init__.py", line 349, in get
"calendar", "v3", http=http, cache_discovery=False
File "/usr/local/lib/python3.7/site-packages/oauth2client/_helpers.py", line 133, in positional_wrapper
return wrapped(args, *kwargs)
File "/usr/local/lib/python3.7/site-packages/googleapiclient/discovery.py", line 237, in build
raise e
File "/usr/local/lib/python3.7/site-packages/googleapiclient/discovery.py", line 229, in build
requested_url, discovery_http, cache_discovery, cache)
File "/usr/local/lib/python3.7/site-packages/googleapiclient/discovery.py", line 279, in _retrieve_discovery_doc
raise HttpError(resp, content, uri=actual_url)
googleapiclient.errors.HttpError:

[probot-home-assistant[bot]]

google documentation
google source
_{^{(message by IssueLinks)}}

[frenck]

429 = to many requests, often used to indicate you have reached API or account limits.

[ASNNetworks]

429 = to many requests, often used to indicate you have reached API or account limits.

That's strange right? I can't change anything about how much it gets requested right? Any advise what I can do? I created a new Google Calendar API on a different project as I mentioned, but that didn't help. That one has 0 calls.

[der-optimist]

I get the 429 error since 13 hours now, 1 or 2 times every minute. I didn't change anything.
Up to yesterday, everything worked fine. I'm using HA 0.112.4 / Supervisor 228 / Ubuntu 18.04.3 LTS on an Intel NUC

Here is a part of my log:

2020-07-15 13:32:12 ERROR (MainThread) [aiohttp.server] Error handling request Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/aiohttp/web_protocol.py", line 418, in start resp = await task File "/usr/local/lib/python3.7/site-packages/aiohttp/web_app.py", line 458, in _handle resp = await handler(request) File "/usr/local/lib/python3.7/site-packages/aiohttp/web_middlewares.py", line 119, in impl return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/real_ip.py", line 39, in real_ip_middleware return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 73, in ban_middleware return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 127, in auth_middleware return await handler(request) File "/usr/src/homeassistant/homeassistant/components/http/view.py", line 129, in handle result = await result File "/usr/src/homeassistant/homeassistant/components/calendar/__init__.py", line 192, in get request.app["hass"], start_date, end_date File "/usr/src/homeassistant/homeassistant/components/google/calendar.py", line 102, in async_get_events return await self.data.async_get_events(hass, start_date, end_date) File "/usr/src/homeassistant/homeassistant/components/google/calendar.py", line 147, in async_get_events service, params = await hass.async_add_executor_job(self._prepare_query) File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run result = self.fn(*self.args, **self.kwargs) File "/usr/src/homeassistant/homeassistant/components/google/calendar.py", line 132, in _prepare_query service = self.calendar_service.get() File "/usr/src/homeassistant/homeassistant/components/google/__init__.py", line 349, in get "calendar", "v3", http=http, cache_discovery=False File "/usr/local/lib/python3.7/site-packages/oauth2client/_helpers.py", line 133, in positional_wrapper return wrapped(*args, **kwargs) File "/usr/local/lib/python3.7/site-packages/googleapiclient/discovery.py", line 237, in build raise e File "/usr/local/lib/python3.7/site-packages/googleapiclient/discovery.py", line 229, in build requested_url, discovery_http, cache_discovery, cache) File "/usr/local/lib/python3.7/site-packages/googleapiclient/discovery.py", line 279, in _retrieve_discovery_doc raise HttpError(resp, content, uri=actual_url) googleapiclient.errors.HttpError: <HttpError 429 when requesting https://www.googleapis.com/discovery/v1/apis/calendar/v3/rest returned "Too Many Requests"> 2020-07-15 13:32:13 ERROR (MainThread) [homeassistant.helpers.entity] Update for calendar.erinnerungen_bildschirm fails Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 272, in async_update_ha_state await self.async_device_update() File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 466, in async_device_update self.update # type: ignore File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run result = self.fn(*self.args, **self.kwargs) File "/usr/src/homeassistant/homeassistant/components/google/calendar.py", line 106, in update self.data.update() File "/usr/src/homeassistant/homeassistant/util/__init__.py", line 239, in wrapper result = method(*args, **kwargs) File "/usr/src/homeassistant/homeassistant/components/google/calendar.py", line 169, in update service, params = self._prepare_query() File "/usr/src/homeassistant/homeassistant/components/google/calendar.py", line 132, in _prepare_query service = self.calendar_service.get() File "/usr/src/homeassistant/homeassistant/components/google/__init__.py", line 349, in get "calendar", "v3", http=http, cache_discovery=False File "/usr/local/lib/python3.7/site-packages/oauth2client/_helpers.py", line 133, in positional_wrapper return wrapped(*args, **kwargs) File "/usr/local/lib/python3.7/site-packages/googleapiclient/discovery.py", line 237, in build raise e File "/usr/local/lib/python3.7/site-packages/googleapiclient/discovery.py", line 229, in build requested_url, discovery_http, cache_discovery, cache) File "/usr/local/lib/python3.7/site-packages/googleapiclient/discovery.py", line 279, in _retrieve_discovery_doc raise HttpError(resp, content, uri=actual_url) googleapiclient.errors.HttpError: <HttpError 429 when requesting https://www.googleapis.com/discovery/v1/apis/calendar/v3/rest returned "Too Many Requests">

And here is a screenshot of my google console. Doesn't look like too many requests...

[der-optimist]

Sorry for bad format. Here the log again.

2020-07-15 13:32:12 ERROR (MainThread) [aiohttp.server] Error handling request
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/aiohttp/web_protocol.py", line 418, in start
resp = await task
File "/usr/local/lib/python3.7/site-packages/aiohttp/web_app.py", line 458, in _handle
resp = await handler(request)
File "/usr/local/lib/python3.7/site-packages/aiohttp/web_middlewares.py", line 119, in impl
return await handler(request)
File "/usr/src/homeassistant/homeassistant/components/http/real_ip.py", line 39, in real_ip_middleware
return await handler(request)
File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 73, in ban_middleware
return await handler(request)
File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 127, in auth_middleware
return await handler(request)
File "/usr/src/homeassistant/homeassistant/components/http/view.py", line 129, in handle
result = await result
File "/usr/src/homeassistant/homeassistant/components/calendar/__init__.py", line 192, in get
request.app["hass"], start_date, end_date
File "/usr/src/homeassistant/homeassistant/components/google/calendar.py", line 102, in async_get_events
return await self.data.async_get_events(hass, start_date, end_date)
File "/usr/src/homeassistant/homeassistant/components/google/calendar.py", line 147, in async_get_events
service, params = await hass.async_add_executor_job(self._prepare_query)
File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(self.args, *self.kwargs)
File "/usr/src/homeassistant/homeassistant/components/google/calendar.py", line 132, in _prepare_query
service = self.calendar_service.get()
File "/usr/src/homeassistant/homeassistant/components/google/__init__.py", line 349, in get
"calendar", "v3", http=http, cache_discovery=False
File "/usr/local/lib/python3.7/site-packages/oauth2client/_helpers.py", line 133, in positional_wrapper
return wrapped(args, *kwargs)
File "/usr/local/lib/python3.7/site-packages/googleapiclient/discovery.py", line 237, in build
raise e
File "/usr/local/lib/python3.7/site-packages/googleapiclient/discovery.py", line 229, in build
requested_url, discovery_http, cache_discovery, cache)
File "/usr/local/lib/python3.7/site-packages/googleapiclient/discovery.py", line 279, in _retrieve_discovery_doc
raise HttpError(resp, content, uri=actual_url)
googleapiclient.errors.HttpError:
2020-07-15 13:32:13 ERROR (MainThread) [homeassistant.helpers.entity] Update for calendar.erinnerungen_bildschirm fails
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 272, in async_update_ha_state
await self.async_device_update()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 466, in async_device_update
self.update # type: ignore
File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(self.args, *self.kwargs)
File "/usr/src/homeassistant/homeassistant/components/google/calendar.py", line 106, in update
self.data.update()
File "/usr/src/homeassistant/homeassistant/util/__init__.py", line 239, in wrapper
result = method(args, *kwargs)
File "/usr/src/homeassistant/homeassistant/components/google/calendar.py", line 169, in update
service, params = self._prepare_query()
File "/usr/src/homeassistant/homeassistant/components/google/calendar.py", line 132, in _prepare_query
service = self.calendar_service.get()
File "/usr/src/homeassistant/homeassistant/components/google/__init__.py", line 349, in get
"calendar", "v3", http=http, cache_discovery=False
File "/usr/local/lib/python3.7/site-packages/oauth2client/_helpers.py", line 133, in positional_wrapper
return wrapped(args, *kwargs)
File "/usr/local/lib/python3.7/site-packages/googleapiclient/discovery.py", line 237, in build
raise e
File "/usr/local/lib/python3.7/site-packages/googleapiclient/discovery.py", line 229, in build
requested_url, discovery_http, cache_discovery, cache)
File "/usr/local/lib/python3.7/site-packages/googleapiclient/discovery.py", line 279, in _retrieve_discovery_doc
raise HttpError(resp, content, uri=actual_url)
googleapiclient.errors.HttpError:

[edif30]

Happens to me as well. I even created a new API on an entirely different account. I also deleted the .google_token and upon a fresh API, newly rebooted HA with new client secret and ID information, the same result happens.

2020-07-15 08:53:52 ERROR (MainThread) [homeassistant.components.calendar] google: Error on device update!
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 319, in _async_add_entity
    await entity.async_device_update(warning=False)
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 466, in async_device_update
    self.update  # type: ignore
  File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/homeassistant/homeassistant/components/google/calendar.py", line 106, in update
    self.data.update()
  File "/usr/src/homeassistant/homeassistant/util/__init__.py", line 239, in wrapper
    result = method(*args, **kwargs)
  File "/usr/src/homeassistant/homeassistant/components/google/calendar.py", line 169, in update
    service, params = self._prepare_query()
  File "/usr/src/homeassistant/homeassistant/components/google/calendar.py", line 132, in _prepare_query
    service = self.calendar_service.get()
  File "/usr/src/homeassistant/homeassistant/components/google/__init__.py", line 349, in get
    "calendar", "v3", http=http, cache_discovery=False
  File "/usr/local/lib/python3.7/site-packages/oauth2client/_helpers.py", line 133, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/googleapiclient/discovery.py", line 237, in build
    raise e
  File "/usr/local/lib/python3.7/site-packages/googleapiclient/discovery.py", line 229, in build
    requested_url, discovery_http, cache_discovery, cache)
  File "/usr/local/lib/python3.7/site-packages/googleapiclient/discovery.py", line 279, in _retrieve_discovery_doc
    raise HttpError(resp, content, uri=actual_url)
googleapiclient.errors.HttpError: <HttpError 429 when requesting https://www.googleapis.com/discovery/v1/apis/calendar/v3/rest returned "Too Many Requests">

I am running the latest container version 112.5. @ASNNetworks I did everything you did to triage and I get the same exact results.

[ASNNetworks]

I removed all projects from Google Developer except one (that I strictly use for Google Assistant). I then created completeley new one and followed the steps. All goes will even after authenticating. However, HA still does not create the google_calendars.yaml. Either something is seriously wrong/changed or there is an issue withing my account. Only one thing left for me to test: create a new Google account and a clean install of HA to test it. Don't have the time right now, so I hope more people confirm this and it's not something with my setup or Google account.

Log shows the same error.

2020-07-15 14:58:06 ERROR (MainThread) [homeassistant.core] Error executing service: <ServiceCall google.scan_for_calendars (c:9b2e6ab8b0894112ad6955d5473ab292)> Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/core.py", line 1276, in catch_exceptions await coro_or_task File "/usr/src/homeassistant/homeassistant/core.py", line 1299, in _execute_service await self._hass.async_add_executor_job(handler.func, service_call) File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run result = self.fn(*self.args, **self.kwargs) File "/usr/src/homeassistant/homeassistant/components/google/__init__.py", line 257, in _scan_for_calendars service = calendar_service.get() File "/usr/src/homeassistant/homeassistant/components/google/__init__.py", line 349, in get "calendar", "v3", http=http, cache_discovery=False File "/usr/local/lib/python3.7/site-packages/oauth2client/_helpers.py", line 133, in positional_wrapper return wrapped(*args, **kwargs) File "/usr/local/lib/python3.7/site-packages/googleapiclient/discovery.py", line 237, in build raise e File "/usr/local/lib/python3.7/site-packages/googleapiclient/discovery.py", line 229, in build requested_url, discovery_http, cache_discovery, cache) File "/usr/local/lib/python3.7/site-packages/googleapiclient/discovery.py", line 279, in _retrieve_discovery_doc raise HttpError(resp, content, uri=actual_url) **googleapiclient.errors.HttpError: <HttpError 429 when requesting https://www.googleapis.com/discovery/v1/apis/calendar/v3/rest returned "Too Many Requests">**

[edif30]

@ASNNetworks don't bother doing a whole new account. I tried with an entirely separate account, new project/API, and completely clean new install of HA. Same issue. This seems to be an issue on either Googles side or with the way the integration is being called.

[ASNNetworks]

@edif30 thanks for your quick reply, you save me a ton of time and headaches. In the meantime I'm looking at other ways to use Google Calendar (by CalDav or iCal) but that doesn't seem to work so much.

I hope a lot of people experience this and come forward.

Edit: so what's weird as well: I decided to disable the API to see what HA logs would then do. With the API disable it has the exact same error. How is that even possible? I figured since the API is disabled, it shouldn't even be possible to do a request, let a lone too many.

[DubhAd]

I've got the same error in HA, and no errors or API limits reached (or even approached) on the Google end. I'd agree this looks to be a thing on Google's end.

[ciechompl]

I have the same ...

googleapiclient.errors.HttpError: <HttpError 429 when requesting https://www.googleapis.com/discovery/v1/apis/calendar/v3/rest returned "Too Many Requests">

since yesterday or maybe day before ... I haven't changed anything .. simply stopped working after several months running without any issues.

[Korte68]

I think the problem started around 23 of June. since that time the Calendar api recorded 4xx errors....

[markgaze]

I've just set the calendar integration up too and started getting this error almost immediately, it does seem to be an issue with Google, there's a workaround in this StackOverflow answer but I haven't tried it and not sure if it's viable here anyway. Guess we'll have to wait for an answer on the issue from Google.

[sealdana]

I'm having the same issue as you guys and with the same error logs - I'll post if needed. My issues started last night. This is what I've done as I've been trying to narrow it down. I am using virtualbox so I have multiple snapshots going back a couple months. I created new credentials in google, deleted the token file and and restarted HA several times with no success - I repeated the process about 3 times and still no success. Then I reinstalled atomic calendar a couple times and deleted the token file and still no success. I am running HA 112.4 so I restored virtualbox snapshots going back to HA v98 - no success. Then I downloaded the latest .vdi and restored using HA snapshots and no success. I'm thinking the issue is with Google.
I searched for errors in the google console api for the credentials I use but all reports on goodle indicate no errors.

[lindsaymarkward]

Same here, since yesterday.
It looks very much like something has changed at Google's end to cause this.

[markgaze]

Looks like this has been fixed: https://issuetracker.google.com/issues/159368649#comment64

I'm not getting 429 errors any more so if you restart HA your calendar should be able to use the calendar integration again 😊

[ciechompl]

Looks like this has been fixed: https://issuetracker.google.com/issues/159368649#comment64

I'm not getting 429 errors any more so if you restart HA your calendar should be able to use the calendar integration again 😊

I confirm :) HA restart and all works as it should :)

[ASNNetworks]

Yes, I can confirm: everything works again! Closing the for now :)