Test-infra: Utilize Gubernator view_pr for issue tracking [Release Team - bugs, CI, governance, etc.]

[nikopen]

As part of our investigation on automating issue tracking for the release team's various purposes (https://github.com/kubernetes/sig-release/issues/432), we are considering utilizing the existing test infrastructure (Gubernator / Spyglass) so it can be tightly integrated with existing scripts and queries.

The idea came to me after this Gubernator PR Dashboard, where the following features are needed to fulfill our purposes:

• Search both PRs and issues (currently only PRs)
• Additional filters based on labels by modifying the main GH query on URL & via buttons, like the current Milestone ones.
  examples:

  current: https://k8s-gubernator.appspot.com/pr/all?milestone=v1.14
  could be: https://k8s-gubernator.appspot.com/issues/all?updated:>=2019-02-05+label:kind/failing-test
  or https://k8s-gubernator.appspot.com/pr/all?milestone=v1.14+is:closed+closed:>=2019-02-10
  or https://k8s-gubernator.appspot.com/issues/all?milestone=v1.14+label:kind/bug+label:priority/critical-urgent aliased as bit.ly/114-critical-bugs etc.
  etc.

• Altered / adjustable columns (currently they're hardcoded and some are redundant for us)
• 'Notes' column for manual note taking (needs to keep state somewhere)

Extras:

• dashboard creating depending on project/team (i.e. triage, CI, etc) (this can be either aliased via URLs or hyperlink choices on the dashboard page)
• potential advanced integration for CI failing tests/flakes tracking with Prow
• popup fetching the text of the last comment in the ticket (issue/PR).

As Spyglass is being prepped for replacing Gubernator, I'm wondering what work needs to be done to implement the above.

Prior art:
I made this: http://bit.ly/k8s114-bugtriage

Issues and PRs based on a modifiable GH query are automatically fetched and put into the spreadsheet upon opening it, additional manual buttons exist to refresh all rows on demand. Can be as flexible as a GH query allows (another page shows only issues that are closed in the last 10 days, via

Problems:

• Rows are currently not treated as 'objects', not allowing a manual 'Notes' column for note taking per issue/row
• Queries are static and need to be manually changed over various cycles
• Separate spreadsheets/automation need to be manually created / copypasted for various release team functions (bug tracking, CI tracking, governance tracking, docs tracking...)
• Issues with Gapps rate limiting occassionally turning the sheet pale white

I believe that the above problems can be solved in the best way by utilizing Spyglass.

/cc @Katharine @fejta @spiffxp

[nikopen]

/assign @Katharine @nikopen @spiffxp

[stevekuznetsov]

/area prow/deck
So we have a pretty similar page here: https://prow.k8s.io/pr

Only issue I can think if is we would need to make a stateful app to allow for comments and other constant content from users, etc. We're not super well positioned to support that right now/

[nikopen]

@stevekuznetsov while it does have a GH query input, it's hardly a similar page - layout should be row-like as in the gubernator dashboard for visibility, also 'issues' are not supported at all. Its purpose seems to be a bit different from what we want.

Overall there are various partial implementations of what we would ideally have, with some desirable features scattered through them, i.e. GH query window, but it needs a coordinated effort to create a unified solution for issue tracking.

re:state, only a 'notes' column is needed which would be 10-20 words in total, only on Open issues/PRs, and only on 50-100 at a time, we don't need to 'support state' in general. Produced artifact would be KB sized at most, expiring after something closed (only meant for tracking open stuff, otherwise it should be a comment on the ticket itself), a very minimal state which can be stored on S3 or something. If it's _that_ difficult to have, then it can be an extra feature to be implemented after the main design is done.

/remove-area prow/deck

[stevekuznetsov]

while it does have a GH query input, it's hardly a similar page

Sorry, I didn't mean that the page should support your needs, just that it offers similar behavior -- displaying output from a GitHub query.

we don't need to 'support state' in general.

Storing a small amount of state is no easier in a system that is entirely stateless today, unfortunately. Do we want a(nother) web service that's reaching out to S3 to render pages? Seems like we'd be bending over backwards to avoid a database deployment...

[stevekuznetsov]

Why did you remove the deck label? That was added on purpose ...

[stevekuznetsov]

/area prow/deck

(Spyglass is a part of Deck)

[stevekuznetsov]

/cc @cjwagner

[nikopen]

Ah, sorry on the label!

An extra dependency might be unavoidable, the alternative is using external tooling like Airtable which will be extra hassle to manage and maintain etc as I see it.. as the state needed is small with short retention, there could be various hacks to utilize for that - it's a minor feature but somehow 'important' for some teams.

[Katharine]

So I chatted with @nikopen about this some offline, and to summarise my thoughts on this:

I don't think Prow/Spyglass is the right place to put this functionality. Aside from the statefulness requirement, it's also pretty orthogonal to the job running/viewing goals of Prow, and skews into project management. Gubernator does have some similar functionality, but we explicitly did not want to move that into Prow (and currently have no plan for it at all).

While this functionality _would_ (I think) be appropriate for Gubernator, I don't think it actually makes sense to put it there in practice. Firstly there is our general desire to eventually get rid of Gubernator, so I don't think it's productive to sink more engineering time into it. Aside from that, though, Gubernator's PR dashboard does not actually query GitHub. Instead, Gubernator maintains a complete state of the world in Cloud Datastore, maintains it using GitHub webhooks, and queries against that. This makes sense for the intended use-case, which is maintaining a "needs attention" / "incoming" / "outgoing" list, which is updated on comments, etc. However, it means that arbitrary GitHub querying is hard to do, and given the nature of Cloud Datastore, some queries may be entirely infeasible.

Given that this feature is out of scope for Prow/Deck/Spyglass and architecturally incompatible with how Gubernator actually works, I think it most likely makes more sense to build this as its own application (or just use something like Airtable). My understanding that it's pretty simple: it needs to run a GitHub query, put the results in a table, and attach a small bit of metadata to them. I don't think there's much value to be gained in building this on top of either Prow or Gubernator.

[nikopen]

/close

[k8s-ci-robot]

@nikopen: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.