Test-infra: Update py files to py3 (or remove)

[fejta]

Let's update to python3 and/or replace python libraries we still need with their golang equivalents.

EDIT(spiffxp): took a guess at the list of things to convert by looking for /usr/bin/env python$

• [ ] boskos/janitor/gcp_janitor.py
• [x] experiment/bilge.py (#13777)
• [x] experiment/find_developers.py (#13778)
• [x] experiment/fix_testgrid_config.py (#13787)
• [x] experiment/flakedetector.py (#13783)
• [x] experiment/generate_tests.py (#13784)
• [x] experiment/get_job_pods.py
• [x] experiment/graphql_issue_example.py (#13786)
• [x] experiment/maintenance/*.py (#13782)
• [x] experiment/parse_build_log.py (#13779)
• [x] experiment/prepare_release_branch.py (#13785)
• [x] hack/*.py (ref: https://github.com/kubernetes/test-infra/pull/13385)
• [x] kettle/*.py
• [x] metrics/*.py (ref: https://github.com/kubernetes/test-infra/pull/13368, revert: https://github.com/kubernetes/test-infra/pull/13540)
• [x] testgrid/conformance/upload_e2e.py (#13750)
• [x] triage/*.py (ref: https://github.com/kubernetes/test-infra/pull/13732, https://github.com/kubernetes/test-infra/pull/13380 outstanding)
• [x] velodrome/config.py (https://github.com/kubernetes/test-infra/pull/13383)

And some guesses on stuff we may not want to migrate? Or do something other than migrate?

• ❓ gubernator/ - we only use the PR dashboard here, do we want to bother with this?
• ❓ jenkins/bootstrap.* - we consider bootstrap deprecated in favor of pod utils, should we just migrate to them?
• ❓ scenarios/*.py - scenarios are used as part of bootstrap, see above

[Katharine]

Since bazel largely punts on python toolchains: what version of python 3?

[spiffxp]

/priority important-longterm
Doesn't _have_ to get solved this quarter, but Python 2 goes EOL Jan 2020 https://pythonclock.org/

[spiffxp]

/help

[k8s-ci-robot]

@spiffxp:
This request has been marked as needing help from a contributor.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[clarketm]

/assign @clarketm

[spiffxp]

/milestone v1.16

[spiffxp]

2019-06-01

spiffxp@spiffxp-macbookpro:test-infra ((ba215daf8...) %)$ ag -l "/usr/bin/env python$" | wc -l
      58
spiffxp@spiffxp-macbookpro:test-infra ((ba215daf8...) %)$ for f in $(ag -l "/usr/bin/env python$"); do dirname $f; done | sort | uniq
boskos/janitor
experiment
experiment/maintenance
gubernator
gubernator/github
hack
jenkins
kettle
metrics
scenarios
testgrid/conformance
velodrome

2019-07-01

spiffxp@spiffxp-macbookpro:test-infra ((f655dfbc3...) %)$ ag -l "/usr/bin/env python$" | wc -l
      58
spiffxp@spiffxp-macbookpro:test-infra ((f655dfbc3...) %)$ for f in $(ag -l "/usr/bin/env python$"); do dirname $f; done | sort | uniq
boskos/janitor
experiment
experiment/maintenance
gubernator
gubernator/github
hack
jenkins
kettle
metrics
scenarios
testgrid/conformance
velodrome

2018-08-01

spiffxp@spiffxp-macbookpro:test-infra ((87f6d3555...) %)$ ag -l "/usr/bin/env python$" | wc -l
      53
spiffxp@spiffxp-macbookpro:test-infra ((87f6d3555...) %)$ for f in $(ag -l "/usr/bin/env python$"); do dirname $f; done | sort | uniq
boskos/janitor
experiment
experiment/maintenance
gubernator
gubernator/github
jenkins
kettle
metrics
scenarios
testgrid/conformance

[clarketm]

All migrations are complete!

The one exception from the original list is gcp_janitor.py. Since this script is called explicitly from PY2 code and triggered via PY2 runners (e.g. kubekins), it would require additional modification to those interdependent images, script, and tools - all of which we do not plan to migrate.

If we do decide to migrate gcp_janitor.py, then I think we should upgrade it (along with the interdependent components) in a separate, scoped issue ... or forward it over to the boskos maintainers. (also, I am not sure how much active development there actually is on the janitor, so a migration is likely not warranted IMO).

[clarketm]

/close

[k8s-ci-robot]

@clarketm: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[ixdy]

FYI, the version of gcp_janitor.py in kubernetes-sigs/boskos now uses python3.