Test-infra: Testgrid column headers' meanings are undocumented

[porridge]

Some are pretty mysterious to a newcomer.

Also https://github.com/kubernetes/test-infra/issues/2364 is related.

[BenTheElder]

Date (MM-DD)
Time
Job ID
Test-Infra Commit
Tested Commit

I believe improving this is already on the radar but you can also pretty easily match these fields up if you click through to Gubernator.
/area testgrid
/cc @michelle192837

[porridge]

I think that should read:

Date (MM-DD)
Time
Job ID
Tested Commit
Test-Infra Commit

For example for https://k8s-testgrid.appspot.com/sig-scalability-gce-scale#gce-scale-correctness&width=80 the former row contains e.g. c4d3c57f8 which is a valid kubernetes/kubernetes ID, and the latter row contains 923c0b0f4 9bee77673 74452465f 8f529fcab which are valid kubernetes/test-infra IDs:

porridge@kielonek:~/projects/go/src/github.com/porridge/test-infra$ git log -1 --format=oneline  c4d3c57f8
fatal: ambiguous argument 'c4d3c57f8': unknown revision or path not in the working tree.
[...]
porridge@kielonek:~/projects/go/src/github.com/porridge/test-infra$ for id in 923c0b0f4 9bee77673 74452465f 8f529fcab;do git log -1 --format=oneline $id;done
923c0b0f47b6cf261080c90368a1ba5355033a9e (origin/master, origin/HEAD, k8s/master, master) Merge pull request #5024 from adityakali/m63
9bee77673de9e76b2cdbd3e9838d32861dcff677 Merge pull request #4977 from Random-Liu/fix-cri-containerd-node-e2e
74452465f43502c247589f5e81ba2d298cc1fb61 Merge pull request #4943 from BenTheElder/pass-the-args
8f529fcabf273399f7431fa6c65c0b798fd39812 Merge pull request #4872 from BenTheElder/bootstrap.go-parsing
porridge@kielonek:~/projects/go/src/github.com/porridge/test-infra$ kk
porridge@kielonek:~/projects/go/src/k8s.io/kubernetes$ git log -1 --format=oneline  c4d3c57f8c4d3c57f8b3c70f09cb498694fa3db7e46ca3315 Merge pull request #51447 from sakeven/rm_useless_code
porridge@kielonek:~/projects/go/src/k8s.io/kubernetes$ for id in 923c0b0f4 9bee77673 74452465f 8f529fcab;do git log -1 --format=oneline $id;done
fatal: ambiguous argument '923c0b0f4': unknown revision or path not in the working tree.
[...]
fatal: ambiguous argument '9bee77673': unknown revision or path not in the working tree.
[...]
fatal: ambiguous argument '74452465f': unknown revision or path not in the working tree.
[...]
fatal: ambiguous argument '8f529fcab': unknown revision or path not in the working tree.
[...]
porridge@kielonek:~/projects/go/src/k8s.io/kubernetes$ 

[porridge]

And that brings me to another question: what is the meaning of the empty boxes in the Tested commit row? For example for job IDs 10 through 15 in the attached screenshot. Unknown commit? Should I report this as a separate issue?

[BenTheElder]

I think that should read:

Yes. Infra-Commit is a newish feature, mostly people use the tested commit to bisect errors which you can do by dragging across the tests.

And that brings me to another question: what is the meaning of the empty boxes in the Tested commit row? For example for job IDs 10 through 15 in the attached screenshot. Unknown commit? Should I report this as a separate issue?

I'm not sure, looks like unknown commit, those jobs don't seem to have all of the metadata in started.json or finished.json

/cc @krzyzacy @michelle192837

[krzyzacy]

Date (MM-DD)
Time
Job ID

Those three are always present, the others by default: https://github.com/kubernetes/test-infra/blob/master/testgrid/config/config.yaml#L17-L19

and you can also customize that as https://github.com/kubernetes/test-infra/blob/master/testgrid/config/config.yaml#L660-L666

@michelle192837 seems the custom column header is indeed not documented

[spiffxp]

/assign @michelle192837
yeah fixing this is on the v1.9 sig-testing roadmap

for now, the lo-fi way of reverse engineering it as a human is to check out the testgrid config, or infer from metadata fields that are display in gubernator

[michelle192837]

Yup, will be adding column header description/hints.

Didn't realize there's no documentation on it either; I'll see about adding it.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle stale

[porridge]

/remove-lifecycle stale
/lifecycle frozen

[fejta]

/remove-lifecycle frozen

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[spiffxp]

/remove-lifecycle stale
@fejta @michelle192837 This is still an issue, is there anything we community members can do to help fix this? I cringe everytime I demo this and have to explain what all of these columns are by clicking through to gubernator and doing guesswork

[michelle192837]

Not at the moment from the community, sorry, besides poking us to fix it. (Or rather, by the time it's fixable by the community I'd hope it's fixed). That said, I'll put this on my radar for the next two weeks; please feel free to ping me if I haven't updated by then.

[michelle192837]

I have a change on the way to add tooltips on hover for the column headers; should be in next week if nothing goes wrong on release.

[michelle192837]

These are now in! Column headers will display the name of the value the header is based on, and the value itself (useful if, for instance, you're viewing things in Compact or Super Compact size).

[BenTheElder]

Thanks @michelle192837 ! 🎉

[krzyzacy]

:100:

[spiffxp]

This is so close. There are two things I'm fine glossing over as a human, but they might trip up new users:

• Hovering over the build id calls it a Commit
• What does "Infra-Commit" mean? (or, put another way, what repo is "Commit" for, and what repo is "Infra-Commit" for?)

[michelle192837]

Good points. Both of those are based on the configuration_value in column headers (https://github.com/kubernetes/test-infra/blob/18bb81c73264f14d85b2c55bd0d6c93757505bc7/testgrid/config.yaml#L17-L19). Though to be fair, the 'Commit' one is hardcoded in anyways (which I think can be fixed?)

So TestGrid's just pulling that name and those values from the data. Whatever code is actually adding those values to the metadata can be updated to better reflect that. (I think infra-commit is bootstrap.py, but I'm not 100% on that; and I think commit is coming from trying to parse a commit number from either finished.json or started.json, i.e. 'repo-commit'?).

[spiffxp]

/milestone 1.12
Yup. I just don't grok why the build id is also showing up as Commit on mouse hover. It's not explicitly called out in the default column_header.

The bootstrap.py thing was a fun discovery. Guess that means we're not quite able to rip it out in favor of pod utils just yet.

Can configuration_values have spaces in them, or do they have to be dashed-together or underscored_together?

[michelle192837]

I thiiiink they can have spaces, but it's safer if they have dashes/underscores. (i.e. I believe it's legal for JSON and protobuf to have spaces for keys/strings respectively, but it feels a bit weird)

ETA: Wait of course they can have spaces :/ Yeah, that's likely fine unless I'm missing a part in the code that does some weird parsing. That said, dashes/underscores are still probably safer.

[spiffxp]

/milestone v1.13

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[idealhack]

/remove-lifecycle stale

[spiffxp]

/milestone clear

It sure would be nice if we could do this in v1.14, but I'm going to defer to @michelle192837 to add back in to v1.14 milestone

[spiffxp]

/milestone 2019-goals

I think it's a safe 2019 commitment at least

[michelle192837]

Is there a remaining task here, or can it be closed?

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[michelle192837]

/close

[k8s-ci-robot]

@michelle192837: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.