Cloud-builders: Is it possible to use this gcloud docker image for `gcloud app deploy` ?

[giripremise]

I tried to and i get "Request had insufficient authentication scopes." It seems that like when requesting scopes for the builder service account for the project, we request the following scopes:

{"aliases":["default"],"email":"[email protected]","scopes":"https://www.googleapis.com/auth/logging.write\nhttps://www.googleapis.com/auth/projecthosting\nhttps://www.googleapis.com/auth/pubsub\nhttps://www.googleapis.com/auth/devstorage.read_write"}

However in order to deploy we need:

scope="https://www.googleapis.com/auth/appengine.admin https://www.googleapis.com/auth/cloud-platform.read-only https://www.googleapis.com/auth/cloud-platform"

Any suggestions on how i could get around this ?

[skelterjohn]

For the moment, we don't provide the right scopes. We're working on it, and
we hope to have this issue fixed soon. I'll update this issue when things
are ready.

On Tue, Apr 4, 2017, 8:53 PM giripremise [e-mail] wrote:

I tried to and i get "Request had insufficient authentication scopes." It
seems that like when requesting scopes for the builder service account for
the project, we request the following scopes:

{"aliases":["default"],"email":"[email protected]
","scopes":"
https://www.googleapis.com/auth/logging.write\nhttps://www.googleapis.com/auth/projecthosting\nhttps://www.googleapis.com/auth/pubsub\nhttps://www.googleapis.com/auth/devstorage.read_write
"}

However in order to deploy we need:

scope="https://www.googleapis.com/auth/appengine.admin
https://www.googleapis.com/auth/cloud-platform.read-only
https://www.googleapis.com/auth/cloud-platform"

Any suggestions on how i could get around this ?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/GoogleCloudPlatform/cloud-builders/issues/36, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAUZ1kn27q18gthbltKD9eruDG3IAIQAks5rsuX3gaJpZM4MznSz
.

[giripremise]

Thank you. In the meanwhile do you have any suggestions on how i could deploy to flex a docker image that i built using Cloud Builder ?

[skelterjohn]

Yes, there is an --image-url flag for the deploy command. Pass your
gcr.io/foo/bar image as its value. I think the image you use needs to
listen on :8080.

On Tue, Apr 4, 2017, 10:41 PM giripremise [e-mail] wrote:

Thank you. In the meanwhile do you have any suggestions on how i could
deploy to flex a docker image that i built using Cloud Builder ?

—
You are receiving this because you commented.

Reply to this email directly, view it on GitHub
https://github.com/GoogleCloudPlatform/cloud-builders/issues/36#issuecomment-291726060,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAUZ1o1Kn4pRf7oNgAib3HBPYq-THVM8ks5rsv8_gaJpZM4MznSz
.

[giripremise]

My apologies for not being clear.

• Basically i have git build triggers via Cloud Builder that build my image on every checkin to master.
• However once the image to GCR is built i also want to deploy the just built image to flex as part of the build process, sort of continuous deployment.
• I was using the gcloud docker image to achieve this, and ran into the aforementioned scope issue.

So while the scope issue is being fixed, is there any other option so that i could call gcloud app deploy --image-url myimage from my cloudbuild.yaml ?

Let me know if this makes sense.

[skelterjohn]

I see.

You could store a json key for a service account in GCS, and using the
gcloud builder run a script that activates that key and runs 'gcloud app
deploy'.

- name: 'gcr.io/cloud-builders/gsutil'
  args: ['cp', 'gs://your_bucket/key.json', '.']
- name: 'gcr.io/cloud-builders/gcloud'
  entrypoint: 'bash'
  args:
  - -c
  - |
    gcloud auth activate-service-account --key-file key.json
    gcloud app deploy --image-url=foo

It's a bit risky since you're leaving credentials hanging around. We want
to provide better support for this as soon as we can.

On Tue, Apr 4, 2017, 10:53 PM giripremise [e-mail] wrote:

My apologies for not being clear.

• Basically i have git build triggers that build my image on every
  checkin to master.
• However once the image is built i also want to deploy the just built
  image to flex as part of the build process, sort of continuous deployment.
• I was using the gcloud docker image to achieve this, and ran into
  the aforementioned scope issue.

So while the scope issue is being fixed, is there any other option so that
i could call gcloud app deploy --image-url myimage from my
cloudbuild.yaml ?

Let me know if this makes sense.

—
You are receiving this because you commented.

Reply to this email directly, view it on GitHub
https://github.com/GoogleCloudPlatform/cloud-builders/issues/36#issuecomment-291729843,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAUZ1kss-oXTY_MmT2UFdOVdgqYxt34iks5rswIzgaJpZM4MznSz
.

[giripremise]

Thank you

[torbjornvatn]

I hit the same issue trying to use the gcloud image to deploy a cloud image, so I'll be eagerly awaiting this fix as well. (hoping that it has the correct scopes for deploying functions)
Will try the alternative method using another service account in the meantime.

[skelterjohn]

@torbjornvatn We will give it the scopes to do anything you like. If you ever want to limit the abilities of your builder service account, you will be able to do so in the Cloud Console IAM section.

[jmdobry]

Any update on this? I also want to run gcloud app deploy inside a Cloud Container build.

[skelterjohn]

Work is ongoing. We're beholden to a slow-moving internal system.

[imjasonh]

This just recently became possible. You will need to grant permission to deploy to App Engine to your project's builder service account (named like <project-number>@cloudbuild.gserviceaccount). It has the Cloud Container Builder role by default, but you can add App Engine Deployer and try running gcloud app deploy in your build.

Please let us know if you have any trouble, as this is brand new functionality. We'll be updating documentation soon.

[Fabian-K]

It works fine, thanks!

[tkaymak]

@ImJasonH - I don't see the <project-number>@cloudbuild.gserviceaccount in my IAM console.
(There is no account with the any xxx<project-number>@cloudbuild.gserviceaccount) I've tried disabling and enabling the Container Builder API but it's still missing. Can I create it manually?

[skelterjohn]

@james-woods Are you able to create successful builds? If so, [email protected] is definitely listed in your IAM permissions (otherwise it wouldn't work at all). Are you sure you're looking at the right page? It should be listed under https://pantheon.corp.google.com/iam-admin/iam/project?project=$PROJECT_ID if you insert your own project ID.

[tkaymak]

@skelterjohn you are correct, its under IAM permissions, not under service accounts. thank you!

[skelterjohn]

That explains the confusion. This service account is owned by the Cloud Build team, not your project, so it won't show up as one of your service accounts. We give it permissions on your project when you enable the API, so it shows up in IAM.

[tkaymak]

@skelterjohn one more question: I am doing Tests against BigQuery and granted BigQuery permissions to the cloudbuild account. But it still fails with

Error code 404 trying to get security access token from Compute Engine metadata for the default service account. This may be because the virtual machine instance does not have permission scopes specified.

do you have any idea which permission is missing?

[skelterjohn]

For App Engine and gcloud app deploy? There is an IAM role called "App Engine Deployer".

[tkaymak]

Well no, sorry this is a different issue then. I am calling the BigQuery API in a maven test, I wanted to use CloudBuilder as a continuous integration / test automation tool.

[bendory]

Have you already set up the IAM permissions on your Container Builder Service Account per the Big Query Access Control docs? We do not enable any Big Query access whatsoever by default, so until you grant these permissions, you won't be able to use Big Query in your builds.

[imjasonh]

Googling that error message indicates it comes from a Ruby auth library,
NO_METADATA_SERVER_ERROR at
http://www.rubydoc.info/gems/googleauth/0.4.1/Google/Auth

There's a known issue with how some language libraries (Ruby among them)
interact with the metadata server in the Container Builder build
environment specifically, which we hope to address soon.

I don't believe this is related to the specific IAM permissions assigned to
the builder service account, based on the error message.
On Mon, Jun 12, 2017 at 7:05 PM David Bendory [e-mail]
wrote:

Have you already set up the IAM permissions on your Container Builder
Service Account
https://cloud.google.com/container-builder/docs/how-to/service-account-permissions
per the Big Query Access Control docs
https://cloud.google.com/bigquery/docs/access-control? We do not enable
any Big Query access whatsoever by default, so until you grant these
permissions, you won't be able to use Big Query in your builds.

—
You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
https://github.com/GoogleCloudPlatform/cloud-builders/issues/36#issuecomment-307959706,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAM3McuWgCkDzoZwkIIDEqbyArexeLSFks5sDcQqgaJpZM4MznSz
.

[tkaymak]

@bendory yes I actually even tested to give it every permission available but without success. I still get the

Error code 404 trying to get security access token from Compute Engine metadata for the default service account. This may be because the virtual machine instance does not have permission scopes specified.

for tests working with the Dataflow, BigQuery and Storage API. I have tested with a buildstep (gcloud info that the right service account is being used). Anything else I could try?

[bendory]

@james-woods see ^^^@ImJasonH's comment regarding a known metadata issue that we hope to resolve soon.

[bbhoss]

Is there a link to this issue I can monitor? I'm trying to access cloud dataflow and bigquery and apparently my code can't see any configuration. gcloud info shows my service account and a path to the config_default, but it's blank. I opened a ticket because this seems like a bug, but now I'm reading this is a known issue? This behavior/limitation should be documented so people don't spend multiple days trying to figure out what is an undocumented limitation of GCP Cloud Builder.

[imjasonh]

This issue seems to have careened through a few seemingly related auth issues. The good news is the original bug (can't run gcloud app deploy inside a build) is fixed now, if you give your builder service account the correct roles. See https://cloud.google.com/container-builder/docs/securing-builds/set-service-account-permissions#granting_additional_access

As for the gcloud config_default issue you're seeing, is that causing any problems/breakages in your build, or is it just a confusing behavior?

[bbhoss]

I have an issue open here fully describing the problem I'm having, which seems very related to this one. It doesn't seem like support understands what I'm talking about though.

[imjasonh]

Thanks, I'll reply on that issue.