Pipeline: We aren't running bucket storage-related e2e tests in CI

[abayer]

Expected Behavior

We should be running all our e2e tests.

Actual Behavior

We're not! TestStorageBucketPipelineRun and TestStorageTaskRun get skipped because there's no GCP_SERVICE_ACCOUNT_KEY_PATH env var set. As a result, issues like #819 and #820 just get missed.

Steps to Reproduce the Problem

1. check the e2e test results. See that we're not running those tests.

Additional Info

@bobcatfish @dlorenc - do we have creds we could put on the relevant containers in our test execution?

[bobcatfish]

tee hee i guess ill close https://github.com/tektoncd/pipeline/issues/791 as a duplicate :D

[abayer]

whoops! I was in such a frenzy of opening tickets that I didn't look for an existing one. =)

[sbwsg]

/assign @sbwsg

[bobcatfish]

We still need to fix the output -> input bucket linking test. @sbwsg do you still want to take this on or should we unassign you for now?

[sbwsg]

I'm still interested in solving this but likely won't get to it until mid next week. I'll unassign myself until then in case anyone wants it resolved ASAP.

[sbwsg]

The fix we introduced for TestStorageTaskRun in #829 unfortunately breaks users who are running the e2e tests in non-GCP environments. Need to add a way to limit tests that rely on specific providers like this.

[ImJasonH]

There are e2e tests for build-gcs storage resources, and more are coming in #1200. Because they only pull publicly-visible objects, no creds are needed and they can be expressed as YAML tests instead of Go tests.

I tried to write a similar YAML test for a "regular" gcs resource pulling the same archive.zip file used in those build-gcs tests, and it fails:

apiVersion: tekton.dev/v1alpha1
kind: TaskRun
metadata:
  name: gcs-resource
spec:
  taskSpec:
    inputs:
      resources:
      - name: source
        type: storage
    steps:
    - image: ubuntu
      command: ['bash', '-c', 'unzip source/archive.zip && cat source/file.txt']
  inputs:
    resources:
    - name: source
      resourceSpec:
        type: storage
        params:
          - name: location
            value: gs://build-crd-tests/archive.zip
          - name: type
            value: gcs

$ kubectl -n default logs gcs-resource-pod-d8007f -c step-fetch-source-pxpt9
...
Error executing command \"gsutil cp gs://build-crd-tests/archive.zip /workspace/source\" ; error exit status 1; cmd_output ERROR: (gsutil) timed out\nThis may be due to network connectivity issues. Please check your network settings, and the status of the service you are trying to reach.

FWIW I'm able to gsutil cp that file just fine from my laptop. build-gcs resource images don't use gsutil, they use the GCS API directly, so I don't know if that's somehow a culprit.