Django-storages: GS_CREDENTIALS clarification

[camflan]

I've downloaded the service account json file and have set GS_CREDENTIALS to both the content of the file directly, as well as the file path. Neither method works, they both reference this URL, which isn't much help either.

Looking at the source isn't much help either since the credentials are passed down to google.auth, which doesn't appear to load the service-account json file itself either.

Can anyone clarify how to specify GS_CREDENTIALS properly?

[camflan]

I ended up setting the GOOGLE_APPLICATION_CREDENTIALS env instead of using GS_CREDENTIALS. Is this the preferred way?

[sww314]

I had the same problem.

For the next guy:

1. Create a service account. https://cloud.google.com/docs/authentication/getting-started
2. Create the key and download XXX.json file.
3. Set an environment variable of GOOGLE_APPLICATION_CREDENTIALS to path of the json file.
4. Make sure your service account has access to the bucket. https://cloud.google.com/storage/docs/access-control/using-iam-permissions

[sww314]

GS_CREDENTIALS does not work. If you set it to the same value as GOOGLE_APPLICATION_CREDENTIALS you get an exception.

Exception Value: This library only supports credentials from google-auth-library-python. See
https://google-cloud-python.readthedocs.io/en/latest/core/auth.html
for help on authentication with this library. /usr/lib/python2.7/site-packages/google/cloud/client.py in __init__, line 123'

[ericpascual]

I have difficulties to have it work. Should GS_CREDENTIALS be set to the content of the XXX.json file or to its path ?
If GS_CREDENTIALS is not the way to proceed, should GOOGLE_APPLICATION_CREDENTIALS be defined as a variable in the Django app settings, or as an env variable ?
I'm trying to have media files stored in a bucket, but can't succeed to have it work although I spent the whole day on this single problem (and I'm not a newbie WRT Django ;).
Thanks in advance for any clue.

[sww314]

GS_CREDENTIALS does nothing anymore... It is ignored by underlying Google library (AFIAK).
Set an environment variable for GOOGLE_APPLICATION_CREDENTIALS to the path of the json file.

[ericpascual]

Many thanks for replying so quickly. I succeeded having it work in the meantime by following this path.

But it means that the json file must be embedded in the Docker image in my case. And BTW, since I have several environments (dev, prod,...), it also means that I have to embed all the credential files in the image (I don't want to have different images) and select the appropriate one at runtime.

Or maybe there is more academic way to proceed. Any suggestion ?

[ericpascual]

_(answering to myself, in case it could help someone hitting the same question)_

After digging in K8S docs (not for the fainted of the heart ;) the answer came out. Embedding the file in the image works of course, but the right way to proceed is to put it in a secret and mount the secret as a volume in the container. The app will see the file as a local one, and setting GOOGLE_APPLICATION_CREDENTIALS its local path will do the final trick.

This way everybody is happier : sensitive data are protected by the secret mechanism and they don't risk to be pushed in some online Git repo by mistake. BTW, it's easy to manage different credentials for different exploitation contexts (dev, qa, prod,...).

HTH

[manufont]

We found another way. Instead of doing this

GS_CREDENTIALS = "path/to/credentials.json"

do this

from google.oauth2 import service_account

GS_CREDENTIALS = service_account.Credentials.from_service_account_file(
    "path/to/credentials.json"
)

[ericpascual]

Thanks for the suggestion. I take note of it.

Is there any benefit for using this method instead of setting GOOGLE_APPLICATION_CREDENTIALS env var ?

[manufont]

The main reason we chose this solution is that it let us centralize every config variable in a single .py file, so we don't have to set every env var in every system that runs the project. And if we want to override the variable, we can still read it from os.environ.

[ericpascual]

Makes sense. I'll keep this point in mind and I'll check if it works here too.

However, since is has been said above that GS_CREDENTIALS was no more taken in account by the underlying lib, how do you explain that it works for you ?

[manufont]

GS_CREDENTIALS is still taken in account. google-cloud-python only throws an exception if the provided credentials are not an instance of google.auth.credentials.Credentials:
https://github.com/GoogleCloudPlatform/google-cloud-python/blob/master/core/google/cloud/client.py#L125

[ericpascual]

So it seems that the above comment from sww314 is not 100% accurate then. Thanks for the clarification.

[sww314]

Closed via #533