Google-cloud-python: 'ImportError: No module named pwd' while trying to instantiate a pubsub client

Created on 29 Jul 2016 · 5Comments · Source: googleapis/google-cloud-python

We're getting this error when we try to instantiate a PubSub client in our staging environment. This is strange because this code has worked fine for months until today.

Our staging application id is roobeta001.appspot.com.

Traceback (most recent call last):
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 240, in Handle
    handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
    handler, path, err = LoadObject(self._handler)
  File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 85, in LoadObject
    obj = __import__(path[0])
  File "/base/data/home/apps/s~roobeta001/1.394547738564502684/main.py", line 37, in <module>
    excludes=['/login/'])
  File "/base/data/home/apps/s~roobeta001/1.394547738564502684/./server/middleware/event_middleware.py", line 226, in create
    sink = get_sink()
  File "/base/data/home/apps/s~roobeta001/1.394547738564502684/./server/middleware/event_middleware.py", line 219, in get_sink
    return PubSubSink()
  File "/base/data/home/apps/s~roobeta001/1.394547738564502684/./server/middleware/event_middleware.py", line 108, in __init__
    self.topic = pubsub.Client().topic(topic_name)
  File "/base/data/home/apps/s~roobeta001/1.394547738564502684/server/lib/gcloud/client.py", line 186, in __init__
    Client.__init__(self, credentials=credentials, http=http)
  File "/base/data/home/apps/s~roobeta001/1.394547738564502684/server/lib/gcloud/client.py", line 122, in __init__
    credentials = get_credentials()
  File "/base/data/home/apps/s~roobeta001/1.394547738564502684/server/lib/gcloud/credentials.py", line 82, in get_credentials
    return client.GoogleCredentials.get_application_default()
  File "/base/data/home/apps/s~roobeta001/1.394547738564502684/server/lib/oauth2client/client.py", line 1288, in get_application_default
    return GoogleCredentials._get_implicit_credentials()
  File "/base/data/home/apps/s~roobeta001/1.394547738564502684/server/lib/oauth2client/client.py", line 1273, in _get_implicit_credentials
    credentials = checker()
  File "/base/data/home/apps/s~roobeta001/1.394547738564502684/server/lib/oauth2client/client.py", line 1226, in _implicit_credentials_from_files
    credentials_filename = _get_well_known_file()
  File "/base/data/home/apps/s~roobeta001/1.394547738564502684/server/lib/oauth2client/client.py", line 1392, in _get_well_known_file
    default_config_dir = os.path.join(os.path.expanduser('~'),
  File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/posixpath.py", line 268, in expanduser
    import pwd
ImportError: No module named pwd

bug auth

Source

zenlambda

Most helpful comment

The fix for expanduser should be out in the next release of GAE. In the meantime, you can try to monkeypatch os.path.expanduser in appengine_config.py:

import os.path

def patched_expanduser(path):
    return path

os.path.expanduser = patched_expanduser

theacodes on 29 Jul 2016

👍6

All 5 comments

We also have other application ids but this issue only occurs on roobeta001. Is there a change to the runtime being rolled out?

zenlambda on 29 Jul 2016

Thanks for reporting @zenlambda!

The issue is due to Appengine not supporting python system modules.
This should be fixed soon though, but make sure you have gcloud-python 0.17.0 or higher installed.

See: https://github.com/GoogleCloudPlatform/gcloud-python/issues/1893#issuecomment-233047621

daspecster on 29 Jul 2016

👍1

The fix for expanduser should be out in the next release of GAE. In the meantime, you can try to monkeypatch os.path.expanduser in appengine_config.py:

import os.path

def patched_expanduser(path):
    return path

os.path.expanduser = patched_expanduser

theacodes on 29 Jul 2016

👍6

@daspecster Thanks for the quick response and explanation.

@jonparrott Thanks for the workaround. I've tested it in our environment and it seems to work.

zenlambda on 29 Jul 2016

🎉3

Excuse me, I made a mistake when I visited HDDS with kerberos. I said that I didn't have a PWD module, and I saw many solutions for you, but they were unsuccessful. I have wasted two days, no progress, please see the help! Thank you!
*
Traceback (most recent call last):
File "E:/PycharmProjects/TensorFlow/TestEnd/ImpalaTest.py", line 6, in
from krbcontext import krbcontext
File "D:\Anaconda3\envstensorflow\lib\site-packages\krbcontext__init__.py", line 6, in
from .context import krbcontext, krbContext # noqa
File "D:\Anaconda3\envstensorflow\lib\site-packages\krbcontext\context.py", line 21, in
import pwd
ModuleNotFoundError: No module named 'pwd'
*