Sig-release: [REQUEST] remove kubeadm-gce* tests from release blocking

[neolit123]

summary:

the family of tests kubadm-gce*:
https://k8s-testgrid.appspot.com/sig-cluster-lifecycle-all

are a based on a deprecated and unmaintained project called kubernetes-anywhere:
https://github.com/kubernetes/kubernetes-anywhere/

to be able to test such deployments we are using a deployer in test-infra called kubernetes-anywhere
https://github.com/kubernetes/test-infra/blob/master/kubetest/anywhere.go

request:

remove the following tests from release-blocking dashboards:

• kubeadm-gce-master (or a release branch)
• kubeadm-gce-x-on-y (where x and y are k releases)

replace with:

• kubeadm-kind-* (master is done, 1.xx branch tests are to be fixed early next week, hopefully)
• periodic-packages-install-deb

rationale:

• we are testing production grade code using a deprecated deployer!
• the kubeadm conformance coverage is already done by using kind as a deployer:
  https://k8s-testgrid.appspot.com/sig-cluster-lifecycle-all#kubeadm-kind-master
• the kubadm package installation is done using a separate package test:
  https://k8s-testgrid.appspot.com/sig-cluster-lifecycle-all#periodic-packages-install-deb
• due to changes in the e2e testing framework we are now unable to obtain logs from the GCE nodes.
  this means we cannot debug properly the test jobs. there might be a solution for that, but after a number of PRs i gave up.
• kubernetes-anywhere is not reproducible locally! it's a strange beast and works locally only when it wants to (weird SSH failures).

we don't have a replacement for kubeadm-gce-x-on-y yet but it will come in the 1.15 cycle.

stakeholders:

i'm going to add some developers here that have witnessed the horrors of kubernetes-anywhere lately:
@fabriziopandini @krzyzacy @BenTheElder @neolit123

/sig release
/sig cluster-lifecycle
/sig testing

---

/assign @timothysc @spiffxp @neolit123

[neolit123]

/kind cleanup

[timothysc]

Yes please, KILL IT WITH FIRE!!!

[krzyzacy]

/pony agree

[k8s-ci-robot]

@krzyzacy: Couldn't find a pony matching that query.

In response to this:

/pony agree

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[spiffxp]

/milestone v1.14

[spiffxp]

Discussed briefly in today's release-team meeting and sigs.k8s.io/kind meeting. Summarizing discussion with @BenTheElder @dims @neolit123

I'm not interested in just immediately flipping things around and using that as precedent. But I agree with the rationale. I lean in favor of this given that it's the sole job that provides kubeadm coverage, and this ask is generally agreed up by the maintainers of all relevant projects.

There was concern that this will drop coverage of "kubeadm + gcp cloud provider" from release-blocking, but we feel that kind will provide sufficient "kubeadm" coverage, and the existing kube-up jobs provide sufficient "gcp cloud provider" coverage.

My proposal:

• move release-master-blocking/kubeadm-gce-master to release-master-informing
• move release-master-blocking/kubeadm-gce-stable-on-master to release-master-informing
• for release-1.y-blocking variants, move to release-1.y-all (if they aren't already there)
• add kind jobs to release-master-informing
• wait for a week to collect data
• if we find signal looks good, move kind jobs to release-master-blocking

[spiffxp]

/assign @amwat @mariantalla
test-infra and ci-signal changes for 1.14

[spiffxp]

@neolit123 one outstanding question I have is:

• https://testgrid.k8s.io/sig-release-master-blocking#packages-pushed-master
• is this what you mean by periodic-packages-install-deb? are there release-branch variants that need to exist?

[neolit123]

@spiffxp

• move release-master-blocking/kubeadm-gce-master to release-master-informing
• move release-master-blocking/kubeadm-gce-stable-on-master to release-master-informing
• for release-1.y-blocking variants, move to release-1.y-all (if they aren't already there)
• add kind jobs to release-master-informing
• wait for a week to collect data
• if we find signal looks good, move kind jobs to release-master-blocking

thank you. i will start sending the PRs for this change.

• is this what you mean by periodic-packages-install-deb? are there release-branch variants that need to exist?

these two jobs:
https://testgrid.k8s.io/sig-cluster-lifecycle-all#periodic-packages-pushed
https://testgrid.k8s.io/sig-cluster-lifecycle-all#periodic-packages-install-deb
test the whole currently support version skew.

periodic-packages-install-deb also tests the installation of latest CI artifacts.

so we technically don't need release branch variants for these. possibly keeping them in release master blocking is sufficient. except that instead of blocking we might want to move them to release-informing because they can block master due to say the 1.12.10 packages missing.

[justaugustus]

Do we have someone driving this forward for 1.15?
/milestone v1.15
/area release-team release-eng
/priority critical-urgent

[neolit123]

@justaugustus
yes.
the problem jobs are already out of release blocking, but feel free to keep this around as the replacements are not in place yet.

[imkin]

/cc imkin