Google-cloud-python: pip install gcloud causes python to raise errors

[jlowin]

(using a fresh conda env to demonstrate):

conda create -n gcloud-test python=3 -y
source activate gcloud-test
pip install gcloud
python -c "print('hello world')" 

output:

Error processing line 2 of /Users/jlowin/anaconda/envs/gcloud-test/lib/python3.5/site-packages/googleapis_common_protos-1.1.0-py3.5-nspkg.pth:

  Traceback (most recent call last):
    File "/Users/jlowin/anaconda/envs/gcloud-test/lib/python3.5/site.py", line 167, in addpackage
      exec(line)
    File "<string>", line 1, in <module>
  KeyError: 'google'

Remainder of file ignored
hello world

After some trial and error, it looks like the current version of protobuf (3.0.0b2.post1) is causing the problem. Based on the solution to #1304, reverting to a prior version resolves it (or at least suppresses the error):

pip install protobuf==3.0.0b1.post2
python -c "print('hello world')" 

clean up:

source deactivate
conda env remove -n gcloud-test -y

[bendemaree]

Came here to note this as well. This is almost certainly not an issue with this package, but I wasn't sure if the google/protobuf project was the right place either.

Rolling back to 3.0.0b2 fixed it for us. This commit intuitively seems like it could have caused an issue. Looks like 3.0.0b2.post1 broke it.

[bendemaree]

Interestingly, the installation itself is not broken for me from a blank virtualenv:

[I] ~/ > vf tmp
New python executable in /Users/ben/.virtualenvs/tempenv-179f205555c18/bin/python2.7
Also creating executable in /Users/ben/.virtualenvs/tempenv-179f205555c18/bin/python
Please make sure you remove any previous custom paths from your /Users/ben/.pydistutils.cfg file.
Installing setuptools, pip, wheel...done.
[I] (tempenv-179f205555c18) ~/ > pip freeze
wheel==0.24.0
You are using pip version 7.1.2, however version 8.0.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
[I] (tempenv-179f205555c18) ~/ > pip install gcloud
Collecting gcloud
  Using cached gcloud-0.10.1-py2.py3-none-any.whl
Collecting pyOpenSSL (from gcloud)
  Using cached pyOpenSSL-0.15.1-py2.py3-none-any.whl
Collecting six (from gcloud)
  Using cached six-1.10.0-py2.py3-none-any.whl
Collecting httplib2>=0.9.1 (from gcloud)
Collecting oauth2client>=2.0.0.post1 (from gcloud)
Collecting protobuf>=3.0.0b2 (from gcloud)
  Using cached protobuf-3.0.0b2.post1-py2-none-any.whl
Collecting googleapis-common-protos (from gcloud)
Collecting cryptography>=0.7 (from pyOpenSSL->gcloud)
Collecting pyasn1-modules>=0.0.5 (from oauth2client>=2.0.0.post1->gcloud)
  Using cached pyasn1_modules-0.0.8-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from oauth2client>=2.0.0.post1->gcloud)
  Using cached rsa-3.3-py2.py3-none-any.whl
Collecting pyasn1>=0.1.7 (from oauth2client>=2.0.0.post1->gcloud)
  Using cached pyasn1-0.1.9-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): setuptools in /Users/ben/.virtualenvs/tempenv-179f205555c18/lib/python2.7/site-packages (from protobuf>=3.0.0b2->gcloud)
Collecting enum34 (from cryptography>=0.7->pyOpenSSL->gcloud)
Collecting ipaddress (from cryptography>=0.7->pyOpenSSL->gcloud)
  Using cached ipaddress-1.0.16-py27-none-any.whl
Collecting idna>=2.0 (from cryptography>=0.7->pyOpenSSL->gcloud)
  Using cached idna-2.0-py2.py3-none-any.whl
Collecting cffi>=1.4.1 (from cryptography>=0.7->pyOpenSSL->gcloud)
Collecting pycparser (from cffi>=1.4.1->cryptography>=0.7->pyOpenSSL->gcloud)
Installing collected packages: six, enum34, ipaddress, pyasn1, idna, pycparser, cffi, cryptography, pyOpenSSL, httplib2, pyasn1-modules, rsa, oauth2client, protobuf, googleapis-common-protos, gcloud
Successfully installed cffi-1.5.2 cryptography-1.2.3 enum34-1.1.2 gcloud-0.10.1 googleapis-common-protos-1.1.0 httplib2-0.9.2 idna-2.0 ipaddress-1.0.16 oauth2client-2.0.1 protobuf-3.0.0b2.post1 pyOpenSSL-0.15.1 pyasn1-0.1.9 pyasn1-modules-0.0.8 pycparser-2.14 rsa-3.3 six-1.10.0
You are using pip version 7.1.2, however version 8.0.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
[I] (tempenv-179f205555c18) ~/ > pip freeze
Traceback (most recent call last):
  File "/Users/ben/.virtualenvs/tempenv-179f205555c18/bin/../lib/python2.7/site.py", line 703, in <module>
    main()
  File "/Users/ben/.virtualenvs/tempenv-179f205555c18/bin/../lib/python2.7/site.py", line 683, in main
    paths_in_sys = addsitepackages(paths_in_sys)
  File "/Users/ben/.virtualenvs/tempenv-179f205555c18/bin/../lib/python2.7/site.py", line 282, in addsitepackages
    addsitedir(sitedir, known_paths)
  File "/Users/ben/.virtualenvs/tempenv-179f205555c18/bin/../lib/python2.7/site.py", line 204, in addsitedir
    addpackage(sitedir, name, known_paths)
  File "/Users/ben/.virtualenvs/tempenv-179f205555c18/bin/../lib/python2.7/site.py", line 173, in addpackage
    exec(line)
  File "<string>", line 1, in <module>
KeyError: 'google'

[jlowin]

@bendemaree you're right, I think my title was misleading -- the gcloud install actually works, but python (gracefully) fails whenever you run it, as you demonstrated.

[dhermes]

This was "fixed" by #1571

[bgenchel]

neither of the above solutions did anything for me. I installed every distribution of protobuf mentioned and couldn't get rid of the problem. Additionally, I uninstalled gcloud, which was what originally caused the problem to surface, and the problem still didn't go away. The pull request this issue links is of no help. No idea what those people are talking about.

[dhermes]

Can you provide a stacktrace indicating what has failed?

[bgenchel]

I got rid of the error messages by literally going into site packages and moving every problem file and directory into a temp directory elsewhere, but I'm not sure that's proper and if something will go wrong as a result.

[bgenchel]

apologies for the picture link; using tmux and having trouble copying and pasting text from the terminal

[dhermes]

This looks very familiar to #2038. Rather than removing every problem file, you should be able to pip uninstall or conda uninstall.

[dominiek]

Unfortunately, this is still happening. Tried various versions of protobuf and re-installing google cloud.

My environment:

• Ubuntu 14.04
• Not using virtual env, since it runs in a Docker container
• Python version 2.7.6
• Protobuf versions tried: 3.1.0, 3.1.0.post1, 3.0.0.b2.post1, 3.0.0.b2.post2

When using python or pip anywhere, this happens:

root@1b99dfc34423:/workdir# python
Error processing line 2 of /usr/local/lib/python2.7/dist-packages/google_cloud_core-0.20.0-py2.7-nspkg.pth:

  Traceback (most recent call last):
    File "/usr/lib/python2.7/site.py", line 158, in addpackage
      exec line
    File "<string>", line 1, in <module>
  KeyError: 'google'

Remainder of file ignored
Error processing line 2 of /usr/local/lib/python2.7/dist-packages/google_cloud_storage-0.20.0-py2.7-nspkg.pth:

  Traceback (most recent call last):
    File "/usr/lib/python2.7/site.py", line 158, in addpackage
      exec line
    File "<string>", line 1, in <module>
  KeyError: 'google'

Remainder of file ignored
Error processing line 2 of /usr/local/lib/python2.7/dist-packages/googleapis_common_protos-1.3.5-nspkg.pth:

  Traceback (most recent call last):
    File "/usr/lib/python2.7/site.py", line 158, in addpackage
      exec line
    File "<string>", line 1, in <module>
  KeyError: 'google'

Remainder of file ignored
Python 2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

[dhermes]

@dominiek When other users have reported similar, they had an "incorrect" install of some packages that live in the google namespace. The fix was to uninstall those conflicting packages and then re-install fresh.

It seems setuptools / pip couldn't handle the change we made from 0.19.0 to 0.20.0.

Before uninstalling lots of things from your default environment, spin up a virtual environment and see if the problem still persists by running:

(venv) $ pip install --upgrade google-cloud

[theacodes]

Also, note that using docker doesn't obviate the need for virtualenv. You
probably still want your application's packages isolated from Ubuntu's.

On Wed, Oct 19, 2016, 9:56 PM Danny Hermes [email protected] wrote:

@dominiek https://github.com/dominiek When other users have reported
similar, they had an "incorrect" install of some packages that live in the
google namespace. The fix was to uninstall those conflicting packages and
then re-install fresh.

It seems setuptools / pip couldn't handle the change we made from 0.19.0
to 0.20.0.

Before uninstalling lots of things from your default environment, spin up
a virtual environment and see if the problem still persists by running:

(venv) $ pip install --upgrade google-cloud

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/GoogleCloudPlatform/google-cloud-python/issues/1570#issuecomment-255009093,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAPUc3cleHcjYo6uGAn3g9r8vz7_NcFQks5q1vSIgaJpZM4HpA3p
.

[brianrusso]

I encountered this problem and it was caused by having https://pypi.python.org/pypi/google installed.

Removing this package and reinstall'ing google-cloud resolved the issue (I verified this by reinstalling 'google' and the problem regressed).

[dhermes]

@brianrusso Unfortunately we don't have any affiliation with that project.

[namitapg]

I encountered this problem and it was caused by having https://pypi.python.org/pypi/google installed.

Removing this package and reinstall'ing google-cloud resolved the issue (I verified this by reinstalling 'google' and the problem regressed).

How did you uninstall it, my pip or rather any python command is not working :( Suddenly everything stopped working after installing this package. My local app-engine server or normal python command is not working.

Traceback (most recent call last):
  File "/Users/namita/penv/lib/python2.7/site.py", line 703, in <module>
    main()
  File "/Users/namita/penv/lib/python2.7/site.py", line 683, in main
    paths_in_sys = addsitepackages(paths_in_sys)
  File "/Users/namita/penv/lib/python2.7/site.py", line 282, in addsitepackages
    addsitedir(sitedir, known_paths)
  File "/Users/namita/penv/lib/python2.7/site.py", line 204, in addsitedir
    addpackage(sitedir, name, known_paths)
  File "/Users/namita/penv/lib/python2.7/site.py", line 173, in addpackage
    exec(line)
  File "<string>", line 1, in <module>
KeyError: 'google'

Kindly help.

[daspecster]

You did a $ pip install google and it broke your virtualenv? If so, that's interesting.

@namitapg I would try creating a fresh virtualenv to start with. You may be able to manually delete the google package from your current virtualenv's site-packages but I'm not sure that would resolve the issue.

[namitapg]

I did pip install --upgrade google-cloud-storage. Afterwards, nothing was working. Also, I removed that package from site-packages, still did not work. Now, I have set up new virtualenv. And everything is working now :)

Our requirements.txt includes google-cloud==0.22.0. I was getting DistributionNotFound: The 'google-cloud-core' distribution was not found and is required by the application error, so I installed this package.

Thanks for your response.

[daspecster]

Glad to hear it's working!

[vijaypalmanit]

deleting all the files with name and path site-packages/googleapis***
for the folder 'site-packages' worked for me.

[daylenriggs]

I just commented out the print statements in site.py - everything's working fine, and no more annoying errors messages