Kubeadm: Test pinned releases are still accessible

[jamiehannaford]

In https://github.com/kubernetes/kubeadm/issues/250#issuecomment-296178197, a user suggested adding tests to verify that historic releases are still accessible in package managers.

The other part is that you'd like to see fixed-version tests (which I think is achievable by doing something like apt-get install kubeadm=1.5.6) to make sure older versions are still accessible in the repos and working as intended. Does that sound right?

This could be a corollary of #263.

[luxas]

cc @mikedanese @pipejakob who own the apt/yum repos

@jamiehannaford What should this test look like? Where would it live?
Do you see yourself working on it ;)?

[jamiehannaford]

@mikedanese @pipejakob Is it possible to add a new job to prow/config which periodically runs apt/yum commands in distro containers against versioned kubeadm targets?

Something like:

- name: ci-kubeadm-e2e-ubuntu-release
  interval: 24h
  spec:
    containers:
    - args:
      - --timeout=300
      - --bare
      image: ubuntu
      command: apt-get update && apt-get install -y kubeadm-1.4

(just an idea, not sure how feasible)

[pipejakob]

It's certainly possible with some extra work. :-) If anyone is eager to take this on, I'll do my best to help introduce and guide through the test-infra layers, but I don't have bandwidth for this in the near term.

[jamiehannaford]

@pipejakob I can take this on after v1.7 is released

[jamiehannaford]

@pipejakob @jessicaochen Do you have bandwidth to help out with this for 1.9? I'm just wondering what such tests would look like apart from periodically performing apt-get

[luxas]

I have bandwidth to review I think. This should basically be a straightforward script verifying that all the latest 3 releases (in this case all v1.6.x, v1.7.x, v1.8.x) have corresponding debs.
e.g. it will query dl.k8s.io/stable.txt, dl.k8s.io/stable-1.7.txt, and so on, and verify all point releases are there as well.

@jamiehannaford do you have time to write such a script/go program?

[luxas]

(this actually belongs more to #sig-release than kubeadm, but...)

[jamiehannaford]

@luxas I think I'll have some time next week to take this on

[luxas]

Thanks @jamiehannaford!

[krzyzacy]

@jamiehannaford I think you can add a simple script in kubeadm repo, and in prow we can have a periodic job that checks out kubeadm and runs that script.

[jamiehannaford]

@krzyzacy Cool. What roughly needs to be added for prow to check out the kubeadm repo?

[krzyzacy]

for instance, currently we have

- name: ci-kubernetes-e2e-kubeadm-gce
    agent: kubernetes
    spec:
      containers:
      - image: gcr.io/k8s-testimages/e2e-kubeadm:v20171030-e52272c1
        args:
        - "--repo=k8s.io/kubernetes=master"
        - ...

you can also have

- name: ci-kubernetes-e2e-kubeadm-verify-version
    agent: kubernetes
    spec:
      containers:
      - image: gcr.io/k8s-testimages/e2e-kubeadm:v20171030-e52272c1
        args:
        - "--repo=k8s.io/kubernetes-anywhere=master"
        - ...

and in config.json you can run

"ci-kubernetes-e2e-kubeadm-verify-version": {
    "args": [
      "./path/to/your/script.sh"
    ],
    "scenario": "execute",
  },