Pipeline: Workload Identity Support

[dansiviter]

Feature request

To simplify GCP Bucket access, support Workload Identity as it's the Google recommended way to authenticate.

Use case

Simplify GCS bucket access and reduce complexity.

[ImJasonH]

Can you describe what doesn't work about WI today? This may just be a documentation issue.

I wrote some unofficial docs here: https://github.com/ImJasonH/tekton-gke/blob/master/workload-identity.md

[dansiviter]

We had a call with a Google Tekton SME and they stated more work was required. If it's a case of just documentation then it would be great to get something more official.

[codeshaw]

@ImJasonH - The issue is not around deployment of pipelines. Currently the installation directions require a kubernetes secret containing GCP service account key json.

We are currently working with inelegant flows where we:

1. Create a service account
2. Create a service account key associated with it
3. Push that key into vault
4. In a deployment pipeline extract that key from vault, creating it as a file
5. Pushing that file into a named kubernetes secret
6. Telling the tekton backend storage ConfigMap the name of the secret

It would be better to be able to give the ConfigMap a service AccountID that has been pre-configured for WLI, and then most of the above workflow goes away.

[dansiviter]

@ImJasonH For reference, it's this part of the documentation: docs/install.md#example-configuration-for-a-gcs-bucket

[ImJasonH]

Thanks, I'll take a look at those instructions and try to spruce them up to take WI into account.

In the meantime, that configuration is really only necessary to support PipelineResources, which are considered "alpha" and are likely to undergo some changes in the future. You might want to consider not relying on them for now, and instead simulate the same functionality by having a Task that fetches from GCS then a dependent Task that operates on that source, with a shared workspace between them.

The benefit of this is that workspaces don't require any GCS-specific setup, and Tasks should already support WI today, as described in the unofficial docs

cc @sbwsg for artifact-storage and workspaces discussion visibility

[bobcatfish]

( I think @wlynch was looking into this at one point as well )

[bobcatfish]

DID NOT MEAN TO CLOSE 🙏

[coryrc]

Until optional workspaces are available, I think you need to fork the gcs-download Task, remove the credential file stuff, then run your Task with the appropriate serviceaccount.

[sbwsg]

Ah, good catch @coryrc . You might also be able to work around this by just passing an emptyDir: {} workspace for the credentials workspace. I haven't tested this but it appears that the Task YAML shouldn't (🤞 ) break if an empty creds dir is provided.

[coryrc]

Ah, good catch @coryrc . You might also be able to work around this by just passing an emptyDir: {} workspace for the credentials workspace. I haven't tested this but it appears that the Task YAML shouldn't ( ) break if an empty creds dir is provided.

And if it does, I bet a fix is easy and welcome!

[wlynch]

/cc @NavidZ who's been working on https://github.com/tektoncd/pipeline/issues/2316 (which I think is the same issue).