Google-api-python-client: KeyError: 'path' intermittent error

[ivan-aguirre]

We've been facing an intermittent error when trying to use oauth2 services. It was working fine until we deployed a new version on GAE. Notice that we haven't changed anything related to this code or the google libraries versions.

That's the exception:

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "./route/auth.py", line 37, in google_auth_callback
    user = service.userinfo().get().execute()
  File "/usr/local/lib/python2.7/site-packages/googleapiclient/discovery.py", line 1121, in methodResource
    schema=schema)
  File "/usr/local/lib/python2.7/site-packages/googleapiclient/discovery.py", line 1026, in __init__
    self._set_service_methods()
  File "/usr/local/lib/python2.7/site-packages/googleapiclient/discovery.py", line 1061, in _set_service_methods
    self._add_basic_methods(self._resourceDesc, self._rootDesc, self._schema)
  File "/usr/local/lib/python2.7/site-packages/googleapiclient/discovery.py", line 1091, in _add_basic_methods
    methodName, methodDesc, rootDesc, schema)
  File "/usr/local/lib/python2.7/site-packages/googleapiclient/discovery.py", line 712, in createMethod
    maxSize, mediaPathUrl) = _fix_up_method_description(methodDesc, rootDesc, schema)
  File "/usr/local/lib/python2.7/site-packages/googleapiclient/discovery.py", line 580, in _fix_up_method_description
    path_url = method_desc['path']
KeyError: 'path'

Environment details

• OS: Google App Engine Flexible, debian:stretch-slim docker image
• Python version: 2.7.15
• pip version: 19.0.3
  
  
  
  • versions:
    
    google-api-core==1.6.0
    
    google-api-python-client==1.7.7
    
    google-auth==1.6.1
    
    google-auth-httplib2==0.0.3
  
  

Steps to reproduce

That's the code, the error raises from service.userinfo().get().execute() .
The error is intermittent and I haven't figured out an specific scenario.

```python
import httplib2
from googleapiclient.discovery import build
from oauth2client.client import AccessTokenCredentials

credentials = AccessTokenCredentials(,
user_agent=str(request.user_agent))
http = httplib2.Http()
http = credentials.authorize(http)

service = build("oauth2", "v2", http=http, developerKey=)

user = service.userinfo().get().execute()

[ngandhy]

Just started getting this myself. Looks like Google is slowly releasing a breaking change

[AdeyinkaAdegbenro]

This is happening to me too!

[ivan-aguirre]

@ngandhy @AdeyinkaAdegbenro

Which version you are using?

[seyisulu]

@ivan-aguirre

This started with version 1.6.2. An upgrade to the latest version 1.7.8 did not solve it.
We first noticed this on a staging project on GAE but now the error happens on our production instance.

[ngandhy]

@ivan-aguirre
google-api-python-client==1.7.4
google-auth==1.6.1
google-auth-httplib2==0.0.3

Any luck figuring it out? Seems to be intermittent

[salvador-bynd]

Having the same issue!

[salvador-bynd]

Clearing the App engine memcache solved the issue.
Will keep an eye on it.

[ngandhy]

is everyone getting it on service.userinfo().get().execute() ?

[yuri-wisestamp]

Clearing app engine cache fixed it for me. Are we expected to have more of this breaking awesomeness any time soon?

[ivan-aguirre]

That's interesting, my application does not use GAE's memcache.
It may be used by some internal GAE stuff...

[ngandhy]

@yuri-wisestamp - considering @ivan-aguirre and I don't use Appengine, I'm assuming it'll kick up again at some point and def seems to be cache related.

I've tried multiple solutions that help for a bit but starts erroring again within 12-24 hours (again possibly pointing to a cache issue).

The most promising solution that has worked for a few days now was to create my own cache instead of using the built in discovery cache.

class RedisGoogleDiscoveryCache(googleapiclient.discovery_cache.base.Cache):
    def get(self, url):
        rc = redis_conn.get('RDGC:%s' % url)
        if rc:
            return json.loads(rc)
        return None

    def set(self, url, content):
        if content:
            redis_conn.set('RDGC2:%s' % url, json.dumps(content))

 def _get_google_client(refresh_token, library, version):
    credentials = _get_http_credentials(refresh_token)
    return discovery.build(
        library, 
        version=version, 
        credentials=credentials, 
        cache=RedisGoogleDiscoveryCache()
    ) 

Hope this helps some of you!

[ivan-aguirre]

@ngandhy, in fact I use app engine flexible but I don't directly use memcache services from my code.

[busunkim96]

Are any of you folks still encountering this? Does it only occur with the OAuth2 API, or have you seen this happen with other APIs?

[busunkim96]

@ivan-aguirre @ngandhy @yuri-wisestamp @salvador-bynd @AdeyinkaAdegbenro
:wave: Anyone still experiencing this issue?

If you are, could you please let us know which environment you are in? (e.g., App Engine Standard, App Engine Flexible)

If there are no responses by September 10 I will assume the issue has resolved itself and close this issue.

---

Some research on the discovery cache:

Discovery will attempt to pull the discovery document from cache by default . link

The appropriate cache module is autodetected. link. The code looks for memcache (available only in App Engine standard) and falls back to the file based cache if the import fails.

[busunkim96]

Closing as this seems to have resolved itself.

If you are experiencing this problem, please open a new issue referencing this one. Thanks!