Google-cloud-python: AttributeError: 'ClientOptions' object has no attribute 'scopes'

[nadeemhussain]

for version google-cloud-core==1.4.0 , storage client initialisation is failing ,

Steps to reproduce

`

from google.cloud import storage
storage_client = storage.Client()
Traceback (most recent call last):
File "", line 1, in
File "/root/.local/lib/python3.7/site-packages/google/cloud/storage/client.py", line 110, in __init__
project=project, credentials=credentials, _http=_http
File "/root/.local/lib/python3.7/site-packages/google/cloud/client.py", line 250, in __init__
Client.__init__(self, credentials=credentials, client_options=client_options, _http=_http)
File "/root/.local/lib/python3.7/site-packages/google/cloud/client.py", line 143, in __init__
scopes = client_options.scopes or self.SCOPE
AttributeError: 'ClientOptions' object has no attribute 'scopes'

`

• install version python3.7.3
• pip3 freeze output
  google-api-core==1.14.2 google-api-python-client==1.7.11 google-auth==1.14.2 google-auth-httplib2==0.0.3 google-cloud==0.34.0 google-cloud-build==1.0.0 google-cloud-core==1.4.0 google-cloud-kms==1.4.0 google-cloud-monitoring==0.35.0 google-cloud-pubsub==1.4.3 google-cloud-storage==1.28.1 google-compute-engine==2.8.13 google-resumable-media==0.5.0 googleapis-common-protos==1.6.0 grpc-google-iam-v1==0.12.3

this is breaking with the change to google-cloud-core==1.4.0 from google-cloud-core==1.3.0, in which it was working as expected,

[eblanchi]

Confirming it is happening for us too (breaking BigQuery initialisation).

[muscovitebob]

We started getting a very similar error today on some of our automated Airflow processes. Here is a reproducible example.

Our requirements.txt file:

google-cloud-automl==1.0.1
google-api-core==1.16.0
google-auth==1.11.2
google-cloud-bigquery==1.24.0
google-cloud-storage==1.26.0
pandas==1.0.1
pandas-gbq==0.13.1
grpcio==1.29.0
pyarrow==0.17.1

Create a virtualenv and install these requirements. You will notice that at the moment the google-cloud-core==1.4.0 library is installed as a dependency of one or more of these above libraries.

Running this file in the above virtualenv is sufficient to reproduce our error:

import google.auth
from google.cloud import bigquery

# Get service account credentials
service_account_credentials, project_id = google.auth.default()

bq_client = bigquery.Client(
    project=project_id, credentials=service_account_credentials
)

print(bq_client.project)

Output:

.venv/lib/python3.8/site-packages/google/auth/_default.py:69: UserWarning: Your application has authenticated using end user credentials from Google Cloud SDK. We recommend that most server applications use service accounts instead. If your application continues to use end user credentials from Cloud SDK, you might receive a "quota exceeded" or "API not enabled" error. For more information about service accounts, see https://cloud.google.com/docs/authentication/
  warnings.warn(_CLOUD_SDK_CREDENTIALS_WARNING)
Traceback (most recent call last):
  File "reproducer.py", line 7, in <module>
    bq_client = bigquery.Client(
  File ".venv/lib/python3.8/site-packages/google/cloud/bigquery/client.py", line 176, in __init__
    super(Client, self).__init__(
  File "/venv/lib/python3.8/site-packages/google/cloud/client.py", line 250, in __init__
    Client.__init__(self, credentials=credentials, client_options=client_options, _http=_http)
  File "/.venv/lib/python3.8/site-packages/google/cloud/client.py", line 136, in __init__
    if credentials and client_options.credentials_file:
AttributeError: 'ClientOptions' object has no attribute 'credentials_file'

[eblanchi]

Successful workaround for our Composer/Airflow environment was to add: google-cloud-core==1.3.0 in the PyPi packages.

[burnzy]

Can confirm also that google-cloud-core==1.3.0 resolved our problem for Composer/Airflow as well — spent a bit of time troubleshooting this one, thanks @eblanchi for the fix!

[pkla]

Also got this same error today running a query on a BigQuery client within a Conda environment. Running conda install google-cloud-core==1.3.0 fixed the issue for me. Thanks @eblanchi !

[busunkim96]

Apologies for this, it looks like I missed bumping the minimum required google-api-core version in google-cloud-core. See https://github.com/googleapis/python-cloud-core/issues/23. I'm going to close this out in favor of tracking the issue there.

As noted above you can pin to google-cloud-core==1.3.0 or alternatively upgrade both google-cloud-core and google-api-core to the latest versions.

[kalron1812]

I am having same issue with this requirements.txt

setuptools>=40.3
grpcio<=1.27.2
google-api-core<1.17.0
Flask

gevent>=0.13

gunicorn>=19.7.1
numpy>=1.18.0
numpy-financial
scipy>=1.4
pvlib>=0.7
google-cloud-storage==1.28.0
google-cloud-datastore==1.12.0
google-cloud-pubsub
pandas==1.0.5