Test-infra: triage is failing

Created on 4 Sep 2019  路  4Comments  路  Source: kubernetes/test-infra

What happened:

go.k8s.io/triage is stale since 8/23

2379 clusters of 130606 failures (6348 in last day) out of 187997 builds from 8/8/2019, 5:00:08 PM to 8/23/2019, 11:44:39 AM.

https://prow.k8s.io/?job=ci-test-infra-triage

checked a few CI failures and they shared common errors:

Traceback (most recent call last):
  File "summarize.py", line 613, in <module>
    main(parse_args(sys.argv[1:]))
  File "summarize.py", line 593, in main
    annotate_owners(data, builds, owners)
  File "summarize.py", line 519, in annotate_owners
    if builds['%s/%s' % (job_path, build)]['started'] > yesterday:
KeyError: 'gs://kubernetes-jenkins/logs/pull/81856/pull-kubernetes-conformance-kind-ipv6/1165011996727316480'

/area triage
/sig testing
/priority critical-urgent

aretriage kinbug prioritcritical-urgent sitesting

All 4 comments

/assign

I am not 100% certain why the GCS bucket location (gs://kubernetes-jenkins/logs/pull/81856/pull-kubernetes-conformance-kind-ipv6/1165011996727316480) in the failure log does not exist. Maybe it has something to do with the upload location changing after the job was defined (https://github.com/kubernetes/test-infra/commit/4461ade9127c546cb37b6f306203343d177194fc). @BenTheElder - are you familiar enough with triage to say if this is a plausible cause?

Regardless of the cause, triage should not be failing on runtime key errors. Unknown keys should be ignored. I will create a PR with this patch (as mitigation) while we investigate the underlying cause to prevent it in the future.

I'm not familiar enough with triage, I've taken a few small pokes at it in the past but it's been a while.

I can confirm it's fixed. The CI looks green

/close

@roycaihw: Closing this issue.

In response to this:

I can confirm it's fixed. The CI looks green

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Was this page helpful?
0 / 5 - 0 ratings