Test-infra: enable Kubernetes-CSI image publishing

[pohly]

What would you like to be added:

As part of https://github.com/kubernetes/k8s.io/issues/158, gcr.io/k8s-staging-csi was created (https://github.com/kubernetes/k8s.io/issues/158#issuecomment-462047593). What we need now are credentials in a secret that the Kubernetes-CSI Prow jobs (https://github.com/kubernetes/test-infra/tree/master/config/jobs/kubernetes-csi) can use to actually push images into that staging area.

Why is this needed:

Currently, images are still being pushed into quay.io from TravisCI while only testing happens in Prow. We want to move entirely to Prow.

[detiber]

/cc
watching since we'd like to do something similar for cluster-api

[amy]

See slack convo here: https://kubernetes.slack.com/archives/CCK68P2Q2/p1563199619408300?thread_ts=1562332672.163800&cid=CCK68P2Q2

Summary: wg-k8s-infra is onboarding cluster-api as a guinea pig for the promotion process and waiting to see how their adoption goes.

[rajibmitra]

I would love to work on "credentials in a secret" part if no one is working on it currently.

[detiber]

Following up on this thread, there has been work on this area by @Katharine and @thockin to enable GCB builds from Prow.

• https://github.com/kubernetes/k8s.io/pull/356 is a pre-requisite for the required permissions
• A prow job needs to be defined similar to: https://github.com/kubernetes/test-infra/blob/31b4bf7cd817eb19005bb682555da14aab2ebf0d/config/jobs/image-pushing/k8s-staging-cluster-api.yaml
• The repository needs a cloudbuild.yaml file defined similar to: https://github.com/kubernetes-sigs/cluster-api/blob/a2e233cfd8b5a0f4f96f4e9eac3f8e07336483a5/cloudbuild.yaml

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[msau42]

/cc
/lifecycle frozen

[pohly]

Some more documentation has been added on how to do this:

• https://github.com/kubernetes/k8s.io/tree/master/k8s.gcr.io
• https://github.com/kubernetes/test-infra/blob/master/config/jobs/image-pushing/README.md

The question also came up whether "docker buildx" for multiple target architectures can be done there, which is a new requirement (https://github.com/kubernetes-csi/node-driver-registrar/pull/63). So far the answer is "maybe" (https://github.com/kubernetes-csi/node-driver-registrar/pull/63#issuecomment-587728830).

[BenTheElder]

Docker buildx should work fine in GCB or prow.

please do not set qemu / multi-arch / binfmt_misc on prow, as binfmt_misc is kernel global. setting it up on GCB is fine, the underlying VM is disposed after your build.

[dims]

almost there getting the docker buildx in cloudbuild.yaml in https://github.com/kubernetes-csi/node-driver-registrar to work. still needs https://github.com/kubernetes-csi/node-driver-registrar/pull/76 to merge

[pohly]

Discussion continues in https://github.com/kubernetes-csi/csi-release-tools/issues/86

[BenTheElder]

I think this was done? Poke.

[pohly]

Yes. We are keeping that csi-release-tools issue open because we also started tracking multi-architecture image publishing which still needs to be done for some sidecars, but as far as test-infra is concerned, everything is working.

/close

[k8s-ci-robot]

@pohly: Closing this issue.

In response to this:

Yes. We are keeping that csi-release-tools issue open because we also started tracking multi-architecture image publishing which still needs to be done for some sidecars, but as far as test-infra is concerned, everything is working.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.