Google-cloud-ruby: GCS - signed URLs (local development)

[jpaulgs]

Environment details

• OS: OSX
• Ruby version: 2.6.3
• Gem name and version: google-cloud-storage 1.26.1

Steps to reproduce

1. gcloud auth application-default login
2. accept oauth flow in browser
3. run the following code

I've also logged in using gcloud auth login with the same results.

Code example

Replace 'some_bucket_name' with a bucket you control

# List files in a bucket 
irb(main):001:0> require 'google/cloud/storage'
irb(main):002:0> storage = Google::Cloud::Storage.new
irb(main):003:0> bucket = storage.bucket('some_bucket_name')
irb(main):004:0> bucket.files
=> []
# that has worked as the bucket I'm pointing to does not have any files
irb(main):005:0> bucket.signed_url('test.txt', method: 'PUT', expires: 600)
Traceback (most recent call last):
        6: from /Users/jeromepaul/.rbenv/versions/2.6.3/bin/irb:23:in `<main>'
        5: from /Users/jeromepaul/.rbenv/versions/2.6.3/bin/irb:23:in `load'
        4: from /Users/jeromepaul/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/irb-1.2.1/exe/irb:11:in `<top (required)>'
        3: from (irb):7
        2: from /Users/jeromepaul/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/google-cloud-storage-1.26.1/lib/google/cloud/storage/bucket.rb:1551:in `signed_url'
        1: from /Users/jeromepaul/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/google-cloud-storage-1.26.1/lib/google/cloud/storage/file/signer_v2.rb:122:in `signed_url'
Google::Cloud::Storage::SignedUrlUnavailable (Google::Cloud::Storage::SignedUrlUnavailable)

The documentation for application default credentials suggests that this should work: https://cloud.google.com/iam/docs/service-accounts#application_default_credentials

The most common use case is testing code on a local machine, and then moving to a development project in Google Cloud, and then moving to a production project in Google Cloud. Using Application Default Credentials ensures that the service account works seamlessly; when testing on your local machine, it uses a locally-stored service account key, but when running on Compute Engine, it uses the project’s default Compute Engine service account.

I've also looked at https://googleapis.dev/ruby/google-cloud-storage/latest/file.AUTHENTICATION.html#cloud-sdk which again suggests that this should work for local development.

I know from the documentation that a SignedUrlUnavailable error is raised when 'when File#signed_url is unable to generate a URL due to missing credentials needed to create the URL.' However I can't find documentation that indicates how to create the required key.

Can the error class please be updated to point to relevant documentation?

[jpaulgs]

To be honest for now the documentation should really reference: https://cloud.google.com/storage/docs/reference/libraries#setting_up_authentication

[quartzmo]

@jpaulgs Thank you for this valuable feedback. We'll see what we can do to improve this. Much appreciated! I'll try to reproduce your code example as soon as I can.

[chvreddy]

@quartzmo Having the same issue in GKE with workload identity feature

rb(main):005:0> bucket.signed_url('test.txt', method: 'PUT', expires: 600) Traceback (most recent call last): 6: from /Users/jeromepaul/.rbenv/versions/2.6.3/bin/irb:23:in `<main>' 5: from /Users/jeromepaul/.rbenv/versions/2.6.3/bin/irb:23:in `load' 4: from /Users/jeromepaul/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/irb-1.2.1/exe/irb:11:in `<top (required)>' 3: from (irb):7 2: from /Users/jeromepaul/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/google-cloud-storage-1.26.1/lib/google/cloud/storage/bucket.rb:1551:in `signed_url' 1: from /Users/jeromepaul/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/google-cloud-storage-1.26.1/lib/google/cloud/storage/file/signer_v2.rb:122:in `signed_url' Google::Cloud::Storage::SignedUrlUnavailable (Google::Cloud::Storage::SignedUrlUnavailable)

This currently relies on having a private key for the service account that is used to sign requests. It's not clear to me how this would be done while using Workload Identity.

[quartzmo]

@frankyn Any idea about using Signed URL V2 in GKE with Workload Identity?

[frankyn]

Hi @quartzmo, it would need to follow a similar solution for Compute Engine instances IIRC. I thought the Ruby library support IAM SignBlob for Signed URLs*

[quartzmo]

I thought the Ruby library support IAM SignBlob for Signed URLs*

@frankyn You mean this signBlob RPC? I don't remember using it in google-cloud-storage; am I forgetting something? (Also, the docs say that method is deprecated.)

[chvreddy]

@quartzmo @frankyn
tried implementing signBlob but got an error when I ran below from my rails console.

response = service.sign_service_account_blob(name, request_body)

Google::Apis::ClientError: forbidden: Permission iam.serviceAccounts.signBlob is required to perform this operation on service account projects/{project-id}/serviceAccounts/{SA-Email}
But my SA account has Service Account Token Creator
So i am curious what went wrong, can we use signBlob with workload identity or its permission issue from my side.

[chvreddy]

@quartzmo @frankyn

Implemented same in google cloud shell
but got different error

Google::Apis::ClientError: badRequest: Request contains an invalid argument. from /bundle/gems/google-api-client-0.32.1/lib/google/apis/core/http_command.rb:228:incheck_status'
from /bundle/gems/google-api-client-0.32.1/lib/google/apis/core/api_command.rb:117:in check_status' from /bundle/gems/google-api-client-0.32.1/lib/google/apis/core/http_command.rb:193:inprocess_response'
from /bundle/gems/google-api-client-0.32.1/lib/google/apis/core/http_command.rb:310:in execute_once' from /bundle/gems/google-api-client-0.32.1/lib/google/apis/core/http_command.rb:113:inblock (2 levels) in execute'
from /bundle/gems/retriable-3.1.2/lib/retriable.rb:61:in block in retriable' from /bundle/gems/retriable-3.1.2/lib/retriable.rb:56:intimes'
from /bundle/gems/retriable-3.1.2/lib/retriable.rb:56:in retriable' from /bundle/gems/google-api-client-0.32.1/lib/google/apis/core/http_command.rb:110:inblock in execute'
from /bundle/gems/retriable-3.1.2/lib/retriable.rb:61:in block in retriable' from /bundle/gems/retriable-3.1.2/lib/retriable.rb:56:intimes'
from /bundle/gems/retriable-3.1.2/lib/retriable.rb:56:in retriable' from /bundle/gems/google-api-client-0.32.1/lib/google/apis/core/http_command.rb:102:inexecute'
from /bundle/gems/google-api-client-0.32.1/lib/google/apis/core/base_service.rb:360:in execute_or_queue_command' from /bundle/gems/google-api-client-0.32.1/generated/google/apis/iamcredentials_v1/service.rb:155:insign_service_account_blob'
from (irb):22
from /bundle/gems/railties-4.2.11.3/lib/rails/commands/console.rb:110:in start' from /bundle/gems/railties-4.2.11.3/lib/rails/commands/console.rb:9:instart'
from /bundle/gems/railties-4.2.11.3/lib/rails/commands/commands_tasks.rb:68:in console' from /bundle/gems/railties-4.2.11.3/lib/rails/commands/commands_tasks.rb:39:inrun_command!'
from /bundle/gems/railties-4.2.11.3/lib/rails/commands.rb:17:in <top (required)>'

Followed link: https://cloud.google.com/iam/docs/reference/rest/v1/projects.serviceAccounts/signBlob

So i am curious what went wrong, can we use signBlob with workload identity

[quartzmo]

@jpaulgs wrote:

I know from the documentation that a SignedUrlUnavailable error is raised when 'when File#signed_url is unable to generate a URL due to missing credentials needed to create the URL.' However I can't find documentation that indicates how to create the required key.

Regarding this error, please see the answer provided by @dazuma on this Stack Overflow question:

Google App Engine (as well as Google Compute Engine, Kubernetes Engine, and Cloud Run) provides "ambient" credentials associated with the VM or instance being run, but only in the form of OAuth tokens. For most API calls, this is sufficient and convenient.

However, there are a small number of exceptions, and Google Cloud Storage is one of them. Recent Storage clients (including the google-cloud-storage gem) may require a full service account key to support certain calls that involve signed URLs. This full key is not provided automatically by App Engine (or other hosting environments). You need to provide one yourself. So as a previous answer indicated, if you're using Cloud Storage, you may not be able to depend on the "ambient" credentials. Instead, you should create a service account, download a service account key, and make it available to your app (for example, via the ActiveStorage configs, or by setting the GOOGLE_APPLICATION_CREDENTIALS environment variable).

[quartzmo]

@jpaulgs asked:

Can the error class please be updated to point to relevant documentation?

I will open a PR to improve the error detail messages for all SignedUrlUnavailable errors, and also update the documentation for the three public #signed_url methods.

[quartzmo]

Here is the new error message:

irb(main):003:0> bucket.signed_url('test.txt', method: 'PUT', expires: 600)
Traceback (most recent call last):

...

Google::Cloud::Storage::SignedUrlUnavailable (Service account credentials 'issuer (client_email)' is missing. To generate service account credentials see https://cloud.google.com/storage/docs/authentication#service_accounts)

[quartzmo]

@chvreddy I do not believe that you can use Workload Identity with the current signed URLs feature. If you find a way to do it, please post the solution here. Thank you!

[frankyn]

Hi folks, reopening.

I missed follow-ups on this thread, apologies.

There are two issues:

1. Ruby Storage library doesn't have an easy way to pass in a signer to introduce an IAM API call signBlob.
2. (still a little fuzzy) You will need to grant role roles/iam.serviceAccountTokenCreator to the Google Service account binded to your project-id.svc.id.goog[k8s-namespace/ksa-name], e.g. [email protected].This same Google Service account email must be passed into the signed_url method as well using issuer.

I provided a prototype that I used to make a successful request within a GKE cluster using Work Identity. I'm new to this environment but hopefully this can help move the conversation forward.

@quartzmo I created a class IAMSigner to get past using an RSA object, and it would be helpful to have a way to provide a lambda that's called instead of the RSA sign method.

require 'googleauth'
require 'google/cloud/storage'
require 'google/apis/iamcredentials_v1'

IAMCredentials = Google::Apis::IamcredentialsV1 # Alias the module


# The following is a hack to introduce IAMCredentialsService into the Storage library
# without modifying the existing library as a proof of concept.
class IAMSigner
   def initialize(issuer)
     @iam_credentials_client = IAMCredentials::IAMCredentialsService.new
     # Get the environment configured authorization
     scopes =  ['https://www.googleapis.com/auth/cloud-platform',
                'https://www.googleapis.com/auth/compute']
     @iam_credentials_client.authorization = Google::Auth.get_application_default(scopes)
     @issuer = issuer
   end

   def sign(digest, string_to_sign)
      # Ignore digest
      request = {
            "payload": string_to_sign,
      }
      response = @iam_credentials_client.sign_service_account_blob(
        "projects/-/serviceAccounts/#{@issuer}",
        request,
        {}
      )
      response.signed_blob
   end
end

storage = Google::Cloud::Storage.new
storage_expiry_time = 5 * 60 # 5 minutes
bucket_name = "anima-frank"
file_name = "unnamed.jpg"
issuer = "[email protected]"
signing_key = IAMSigner.new issuer

url = storage.signed_url bucket_name, file_name, issuer: issuer,
                         signing_key: signing_key,
                         method: "GET", expires: storage_expiry_time,
                         version: :v4

puts url
````

After potential fixes, I'm thinking about something similar to the following:

```ruby
require 'googleauth'
require 'google/cloud/storage'
require 'google/apis/iamcredentials_v1'

IAMCredentials = Google::Apis::IamcredentialsV1 # Alias the module

issuer_and_signer = lambda do |string_to_sign|
     iam_credentials_client = IAMCredentials::IAMCredentialsService.new
     # Get the environment configured authorization
     scopes =  ['https://www.googleapis.com/auth/cloud-platform',
                'https://www.googleapis.com/auth/compute']
     iam_credentials_client.authorization = Google::Auth.get_application_default(scopes)     
     issuer =  iam_credentials_client.authorization.issuer

     request = {
           "payload": string_to_sign,
     }
     response = @iam_credentials_client.sign_service_account_blob(
       "projects/-/serviceAccounts/#{@issuer}",
       request,
       {}
     )
     [issuer, response.signed_blob]
end

storage = Google::Cloud::Storage.new
storage_expiry_time = 5 * 60 # 5 minutes
bucket_name = "anima-frank"
file_name = "unnamed.jpg"

url = storage.signed_url bucket_name, file_name, issuer_and_signer: issuer_and_signer
                         method: "GET", expires: storage_expiry_time,
                         version: :v4

puts url

[quartzmo]

In the last example, should it be issuer_and_signer, given the order of the array [issuer, response.signed_blob] ?

[frankyn]

Completed example doesn't consider issuer being needed to construct string_to_sign.

Let's follow Go's interface a little bit instead with https://github.com/googleapis/google-cloud-go/issues/1130#issuecomment-484236791

So updated the example, it would be:

### PROTOTYPE CODE AND DOES NOT WORK!!!! #####
require 'googleauth'
require 'google/cloud/storage'
require 'google/apis/iamcredentials_v1'

IAMCredentials = Google::Apis::IamcredentialsV1 # Alias the module

iam_credentials_client = IAMCredentials::IAMCredentialsService.new
# Get the environment configured authorization
scopes =  ['https://www.googleapis.com/auth/cloud-platform',
            'https://www.googleapis.com/auth/compute']
iam_credentials_client.authorization = Google::Auth.get_application_default(scopes)     
issuer =  iam_credentials_client.authorization.issuer

signer = lambda do |string_to_sign|
     request = {
           "payload": string_to_sign,
     }
     response = iam_credentials_client.sign_service_account_blob(
       "projects/-/serviceAccounts/#{issuer}",
       request,
       {}
     )
     response.signed_blob
end

storage = Google::Cloud::Storage.new
storage_expiry_time = 5 * 60 # 5 minutes
bucket_name = "anima-frank"
file_name = "unnamed.jpg"

url = storage.signed_url bucket_name, file_name, issuer: issuer, signer: signer
                         method: "GET", expires: storage_expiry_time,
                         version: :v4

puts url

[quartzmo]

So the only public API or behavior change needed in this last example is that signing_key (or its alias private_key) can be a lambda which when called with string_to_sign, returns the signature?

[frankyn]

That's correct @quartzmo, I think maybe a new alias such as signer may help as well but using signing_key is a good starting point.

[frankyn]

@chvreddy could you confirm if https://github.com/googleapis/google-cloud-ruby/issues/6268#issuecomment-663261941 would help in this case?

[chvreddy]

Thanks @frankyn let me apply this patch and see if we are able to create signed URL's

[frankyn]

Thanks for confirming @chvreddy.
If you're attempting to use one of these, I recommend looking at the following example:
(Warning it's still a prototype)

require 'googleauth'
require 'google/cloud/storage'
require 'google/apis/iamcredentials_v1'

IAMCredentials = Google::Apis::IamcredentialsV1 # Alias the module


# The following is a hack to introduce IAMCredentialsService into the Storage library
# without modifying the existing library as a proof of concept.
class IAMSigner
   def initialize(issuer)
     @iam_credentials_client = IAMCredentials::IAMCredentialsService.new
     # Get the environment configured authorization
     scopes =  ['https://www.googleapis.com/auth/cloud-platform',
                'https://www.googleapis.com/auth/compute']
     @iam_credentials_client.authorization = Google::Auth.get_application_default(scopes)
     @issuer = issuer
   end

   def sign(digest, string_to_sign)
      # Ignore digest
      request = {
            "payload": string_to_sign,
      }
      response = @iam_credentials_client.sign_service_account_blob(
        "projects/-/serviceAccounts/#{@issuer}",
        request,
        {}
      )
      response.signed_blob
   end
end

storage = Google::Cloud::Storage.new
storage_expiry_time = 5 * 60 # 5 minutes
bucket_name = "anima-frank"
file_name = "unnamed.jpg"
issuer = "[email protected]"
signing_key = IAMSigner.new issuer

url = storage.signed_url bucket_name, file_name, issuer: issuer,
                         signing_key: signing_key,
                         method: "GET", expires: storage_expiry_time,
                         version: :v4

puts url

[chvreddy]

Thanks for confirming @chvreddy.
If you're attempting to use one of these, I recommend looking at the following example:
(Warning it's still a prototype)

require 'googleauth'
require 'google/cloud/storage'
require 'google/apis/iamcredentials_v1'

IAMCredentials = Google::Apis::IamcredentialsV1 # Alias the module


# The following is a hack to introduce IAMCredentialsService into the Storage library
# without modifying the existing library as a proof of concept.
class IAMSigner
   def initialize(issuer)
     @iam_credentials_client = IAMCredentials::IAMCredentialsService.new
     # Get the environment configured authorization
     scopes =  ['https://www.googleapis.com/auth/cloud-platform',
                'https://www.googleapis.com/auth/compute']
     @iam_credentials_client.authorization = Google::Auth.get_application_default(scopes)
     @issuer = issuer
   end

   def sign(digest, string_to_sign)
      # Ignore digest
      request = {
            "payload": string_to_sign,
      }
      response = @iam_credentials_client.sign_service_account_blob(
        "projects/-/serviceAccounts/#{@issuer}",
        request,
        {}
      )
      response.signed_blob
   end
end

storage = Google::Cloud::Storage.new
storage_expiry_time = 5 * 60 # 5 minutes
bucket_name = "anima-frank"
file_name = "unnamed.jpg"
issuer = "[email protected]"
signing_key = IAMSigner.new issuer

url = storage.signed_url bucket_name, file_name, issuer: issuer,
                         signing_key: signing_key,
                         method: "GET", expires: storage_expiry_time,
                         version: :v4

puts url

sure @frankyn will apply this patch and see how is it going. Thanks!

[frankyn]

Update add in #7091 is pending release of google-cloud-storage gem.

reopening until gem is released.

[joedicator]

@frankyn 🆒 Are you able to comment when the next version of the gem with the fix will be released?

[frankyn]

Hi @joedicator,

I'm pending input from @quartzmo right now. Open PR with next version is https://github.com/googleapis/google-cloud-ruby/pull/6993

[frankyn]

Short update: @quartzmo is aiming to release the library tomorrow.

Thanks @quartzmo!

[frankyn]

Woot! Thanks @quartzmo!

google-cloud-storage 1.27.0 released
Updated API documentation

Closing this issue, please reopen if there's still open questions. Thank you for your patience on this issue.

[quartzmo]

Great addition to google-cloud-storage, thank you @frankyn!

[quartzmo]

Closed by #7091

[chvreddy]

Thanks @quartzmo @frankyn for the support
@frankyn the monkey patch you provided worked great, Thanks! and @frankyn thanks for providing the upgraded gem quickly.

[frankyn]

Thanks @chvreddy for letting us know, it's really great to hear that you were unblocked!

[stefanahman]

How can Workload Identity be used in conjunction with ActiveStorage - Google Cloud Storage?

[frankyn]

Hi @stefanahman, thanks for raising your question!

Could you start a new issue with more background on your use case to better support you?

[mroach]

@stefanahman Did you end up creating an issue and/or figuring it out?

I implemented this signing code and that works, but now I have an implementation dilemma since Google Cloud Storage is only used in production, not tests or development.

[stefanahman]

@stefanahman Did you end up creating an issue and/or figuring it out?

I implemented this signing code and that works, but now I have an implementation dilemma since Google Cloud Storage is only used in production, not tests or development.

No, sorry. I haven't looked into it more.
But you specify the storage per environment, right? Just specify "Disk" as service for those environments:

test:
  service: Disk
  root: <%= Rails.root.join("tmp/storage") %>

local:
  service: Disk
  root: <%= Rails.root.join("storage") %>