Google-api-python-client: google-api-core distribution not found

[prakashraman]

Hi,

I have installed the library using

pip install --upgrade google-api-python-client

Into my virtual env. Everything is working just fine. Although when I push to production and have my build server install packages from the requirements.txt file, I get the error.

The 'google-api-core' distribution was not found and is required by the application

Although, I can see that the package has been installed (or so the build logs say).

Production Env: Using Seed + Serverless

Any help would be great!

[theacodes]

Can you give us the contents of your requirements.txt?

Can you give more details on how you deploy to the production environment? This sort of issue can happen when pkg_resources doesn't have access to package metadata, which can be caused by strange environments that don't use a real python (virtual) environment, like GAE standard.

[prakashraman]

Hi Jon

So I am using "Serverless" (AWS Lamba) - although for the CI infrastructure/tool I am using "Seed". Below is my requirements.txt

aniso8601==1.3.0
apispec==0.25.4
appnope==0.1.0
awscli==1.11.146
backports.shutil-get-terminal-size==1.0.0
boto3==1.4.8
botocore==1.8.1
cachetools==2.0.1
certifi==2018.1.18
cffi==1.11.2
chardet==3.0.4
click==6.7
colorama==0.3.7
decorator==4.1.2
docutils==0.14
enum34==1.1.6
ez-setup==0.9
facebook-sdk==2.0.0
flasgger==0.8.0
Flask==0.12.2
flask-apispec==0.4.0
Flask-Cors==3.0.3
flask-restplus==0.10.1
functools32==3.2.3.post2
futures==3.2.0
gitdb2==2.0.3
GitPython==2.1.7
google-api-core==0.1.4
google-api-python-client==1.6.5
google-auth==1.3.0
google-cloud-core==0.28.0
google-cloud-language==1.0.0
googleapis-common-protos==1.5.3
grpcio==1.9.1
gunicorn==19.7.1
httplib2==0.10.3
idna==2.6
ipython==5.4.1
ipython-genutils==0.2.0
itsdangerous==0.24
Jinja2==2.10
jmespath==0.9.3
jsonschema==2.6.0
MarkupSafe==1.0
marshmallow==2.13.6
mistune==0.8.1
mongoengine==0.13.0
msgpack==0.5.0
msgpack-python==0.4.8
numpy==1.14.0
oauth2client==4.1.2
pathlib2==2.3.0
pexpect==4.2.1
pickleshare==0.7.4
prompt-toolkit==1.0.15
protobuf==3.5.1
ptyprocess==0.5.2
pyasn1==0.4.2
pyasn1-modules==0.2.1
pycparser==2.18
Pygments==2.2.0
pymongo==3.5.1
python-dateutil==2.6.1
python-slugify==1.2.4
pytz==2017.3
PyYAML==3.12
requests==2.18.4
rsa==3.4.2
s3transfer==0.1.11
scandir==1.5
simplegeneric==0.8.1
six==1.11.0
smmap2==2.0.3
traitlets==4.3.2
Unidecode==0.4.21
uritemplate==3.0.0
urllib3==1.22
virtualenv==15.1.0
wcwidth==0.1.7
webargs==1.8.1
Werkzeug==0.12.2

[theacodes]

Can you share anything about how your application is packaged up for deployment?

[tadly]

Getting the same error.
I'm getting this project as dependency of another one (dialogflow) but I can reproduce the problem.

I'm bootstrapping my dependencies into a specific dir so lets replicate that

$ pip install --prefix ./tmp google-api-core
````

This will now throw an exception: 
```python
import site
import pkg_resources

site.addsitedir('./tmp/lib/python3.6/site-packages')
pkg_resources.get_distribution('google-api-core')

This however works:

import site
site.addsitedir('./tmp/lib/python3.6/site-packages')

import pkg_resources
pkg_resources.get_distribution('google-api-core')

Seems like the import order of pkg_resources is important which I can not control in my case.

Any suggestions?

Edit:
I just found something that works

import site
import importlib
import pkg_resources

site.addsitedir('./tmp/lib/python3.6/site-packages')

importlib.reload(pkg_resources)
pkg_resources.get_distribution('google-api-core')

I don't think this is particularly nice but it gets the job done for now...

[soumak77]

I was having the same issue. After upgrading pip via pip install --upgrade pip, the error went away.

[crwilcox]

In looking into this it seems older versions of serverless had issues that have been resolved in newer versions. If you update the frameworkVersion in serverless.yaml to frameworkVersion: ">=1.0.0 <2.0.0" it should be able to get the latest v1.x.x version of the framework.

Relevant issue: https://github.com/serverless/serverless/issues/5026

[mcdonc]

I'm going to close this issue as @crwilcox seems to have identified the problem and provided a workaround. If this is in error, please let me know, and let me know what more is required. Thanks!

[luisdemarchi]

I am with the same problem, I am with serverless in the last version and using the range of versions.

[tomonodes]

Sourced form serverless #6875

Description

1. What did you do?

   I run pip install -r requirements.txt and all the imports work. I've done this in a isolated virtual environment running Python 3.7. Running sls deploy is fine.

sls invoke -f stagger --log fails because of a missing dependency.

my requirements.txt

boto3==1.10.0
fasttext==0.9.1
google-api-python-client==1.7.11
google-cloud-translate==1.7.0
spacy==2.2.1
Flask==1.1.1
joblib==0.14.0
scikit-learn==0.21.3
https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.2.0/en_core_web_sm-2.2.0.tar.gz

1. What happened?

{
    "errorMessage": "The 'google-cloud-translate' distribution was not found and is required by the application",
    "errorType": "DistributionNotFound",
    "stackTrace": [
        "  File \"/var/lang/lib/python3.7/imp.py\", line 234, in load_module\n    return load_source(name, filename, file)\n",
        "  File \"/var/lang/lib/python3.7/imp.py\", line 171, in load_source\n    module = _load(spec)\n",
        "  File \"<frozen importlib._bootstrap>\", line 696, in _load\n",
        "  File \"<frozen importlib._bootstrap>\", line 677, in _load_unlocked\n",
        "  File \"<frozen importlib._bootstrap_external>\", line 728, in exec_module\n",
        "  File \"<frozen importlib._bootstrap>\", line 219, in _call_with_frames_removed\n",
        "  File \"/var/task/handler.py\", line 14, in <module>\n    from translate import detect_and_translate\n",
        "  File \"/var/task/translate.py\", line 5, in <module>\n    from google.cloud import translate\n",
        "  File \"/tmp/sls-py-req/google/cloud/translate.py\", line 18, in <module>\n    from google.cloud.translate_v2 import __version__\n",
        "  File \"/tmp/sls-py-req/google/cloud/translate_v2/__init__.py\", line 20, in <module>\n    __version__ = get_distribution(\"google-cloud-translate\").version\n",
        "  File \"/tmp/sls-py-req/pkg_resources/__init__.py\", line 481, in get_distribution\n    dist = get_provider(dist)\n",
        "  File \"/tmp/sls-py-req/pkg_resources/__init__.py\", line 357, in get_provider\n    return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0]\n",
        "  File \"/tmp/sls-py-req/pkg_resources/__init__.py\", line 900, in require\n    needed = self.resolve(parse_requirements(requirements))\n",
        "  File \"/tmp/sls-py-req/pkg_resources/__init__.py\", line 786, in resolve\n    raise DistributionNotFound(req, requirers)\n"
    ]
}

1. What should've happened?

Should call a Lambda function on AWS.

1. What's the content of your serverless.yml file?

service: stagger

plugins:
  - serverless-python-requirements
  - serverless-wsgi
  - serverless-offline
custom:
  wsgi:
    app: app.app
    packRequirements: false
  pythonRequirements:
    dockerizePip: non-linux
    zip: true

provider:
  name: aws
  runtime: python3.7
  timeout: 29
  stage: dev
  region: eu-west-1
  apiKeys:
      - myStaggerKey

functions:
  stagger:
    handler: handler.stagger
    events:
      - http:
          path: stagger
          method: post
          private: true

package:
    exclude:
        - venv/**

1. What's the output you get when you use the SLS_DEBUG=* environment variable (e.g. SLS_DEBUG=* serverless deploy)

No errors.

Similar or dependent issues:

• #5026
• Issue on googleapis

[busunkim96]

@tomonodes I think the responder on the original issue recommend you file the issue in https://github.com/serverless/serverless-google-cloudfunctions.

Looks like this issue is related to the Google Cloud Functions provider plugin. Could you re-open in that repository since serverless/serverless only covers the core and the AWS implementation. Thanks!

When you file issues on this repo in the future, please open a new issue rather than posting on a closed issue. Maintainers are more likely to miss posts to closed issues. Thanks!