Test-infra: Plugin to add issue to GitHub Project (similar to /milestone)

[jlewi]

What would you like to be added:

I'd like a plugin similar to the milestone plugin that adds an issue to a GitHub project

e.g.

/project 0.5.0

Would add the issue to the project named 0.5.0

Why is this needed:
Kubeflow is using GitHub projects to manage our releases.

https://github.com/orgs/kubeflow/projects/4

Projects provide a Kanban board that can span repositories. Currently to add issues to a project you need write access to the repo. This is too permissive. We'd like a plugin to set it via the prow bot.

Ideally, we could use a GitHub team to track which users have the permission to set the label.

/kind feature

[nikhita]

Related: https://github.com/kubernetes/test-infra/issues/9925

[stevekuznetsov]

/help

[taragu]

@jlewi I have a question regarding the requirements of this issue. How to determine which column to add the PR? Should I also implement something like /project 0.5.0 Backlog which adds an issue/PR to the Backlog column of project 0.5.0?

Also, looks like PR #10401 has many useful methods that can be utilized for this issue. I'll get started on this but at the same time wait for #10401 to be merged before I merge any of my PRs

[jlewi]

@taragu Support for specifying the column like you mentioned e.g

/project 0.5.0 Backlog 

Would be great. I hadn't thought of that.

Can you also support not specifying a column? I think this is what happens by default if you use the UI. I think if you do this depending on the automation setup for the project it will go into some column like "untriaged" or "todo" where it can then be manually processed and moved to the appropriate card/column.

[stevekuznetsov]

The /project board column syntax seems like a good idea to me @taragu . Agreed that waiting on that other PR would be good, or we can break out the client additions in a separate PR that can merge before #10401

[taragu]

@jlewi sure I can support not specifying a column. There are projects that don't have a To do or untriaged column such as https://github.com/kubernetes-sigs/architecture-tracking/projects/3 . In this case, is adding the issue/PR to the Backlog column acceptable?

@stevekuznetsov that's a great idea. Thanks for the suggestion! I'll work on the client portion first.

[spiffxp]

/milestone v1.14

This sounds like a great idea

[jlewi]

@taragu Adding to Backlog by default seems good to me.

[nikhita]

Removing help-wanted since @taragu is going to work on it (thanks for this btw!!)

/remove-help

[taragu]

@jlewi

• Do we allow adding an issue or PR to a project of another org or repo? For example, adding an issue from Kubernetes org to kubeflow
• Also, can an issue or PR be added to more than one projects?

[jlewi]

@taragu I don't think we have an immediate need for adding issues belonging to one organization to another organization.

• Our projects though are at the org level whereas issues are per repo; does that answer your first question?

• Yes we have issues that belong to multiple projects

[taragu]

@jlewi I see. That answers my question. Adding an issue belongs to a repo in org A to a project in org B is not in the scope of this issue.
For issues that belong to multiple projects, does that mean doing /project clear will remove the issue from all projects? Or do we need to specify the specific project like /project clear 0.4.0

[jlewi]

@taragu
I think we'd like to be able to remove it from specific projects e.g.

/project clear 0.4.0

[taragu]

/assign

[taragu]

@jlewi for the use case of column name not provided in the command (/project 0.5.0), originally I was thinking having a list of default columns so that the issue/PR could be added to any one of those (depends on which one is available in the project). And if none of those pre-defined columns is available, the issue/PR will be added to a random column.

This can be a surprising behavior from a user's point of view. What do you think of making column name required in the command?
cc @stevekuznetsov @cblecker

[jlewi]

Can we replicate the GitHub UI behavior? If just tag an issue into a project I think it will be added to a column based on the project automation setup. Is there a way to replicate this?

I think that is the desired behavior because

• users don't need to know card names
• if project automation is configured issue will go into a triage column as desired.

[taragu]

@jlewi cool! Does that mean adding an issue/PR to a project is simply adding a label? What do you mean by tagging an issue?

[jlewi]

Currently the way I add an issue to a project is via the GitHub UI. Specifically if you go to an issue, below labels there is an option to select projects.

From there I can select project. The UI doesn't let me select which card to add the issue to.

For one of our cards in the project we set it up so all newly added isssues & PRs are automatically added to that card.

I would like the same behavior when doing

/project 0.5.0

i.e. I don't want card to be required; in which case I'd like project automation to determine which card it gets applied to.

[taragu]

@jlewi Ah I see. Thanks for the clarification!

I was looking at Github's API for projects and I wasn't able to find a way to add an issue/PR to a project without specifying a column. I also wasn't able to tell which column is the default column for newly added issues/PRs from the list project columns API.

I'm not sure what a good workaround would be. Specifying the column to add for each project in a config file seems too much. Any suggestions @stevekuznetsov ?

[taragu]

@jlewi I have not found a way to get the default column for project automation. My current implementation will require users to edit the plugins.yaml file for the default set of columns for each org or repo (repo configs override org configs):

project_config:
  project_org_configs:
    kubernetes:
      org_maintainers_team_id: 2460384
      org_maintainers_team_name: kubernetes-milestone-maintainers
      org_columns:
        - To do
        - To triage
        - Backlog
      project_repo_configs:
        kubernetes:
          repo_maintainers_team_id: 2460384
          repo_maintainers_team_name: kubernetes-milestone-maintainers
          repo_columns:
            - To do
            - To triage
            - Backlog
    kubeflow:
      org_maintainers_team_id: 2460384
      org_maintainers_team_name: kubernetes-milestone-maintainers
      org_columns:
        - To do
        - To triage
        - Backlog

The column name is not required (i.e. the command is /project 0.5.0) if at least one of the default columns in the config file is found in the project. If no default column is in the config file for the org/repo of the project, an error will be shown to the user.

Please let me know if this is acceptable. Any suggestions would be appreciated!
@stevekuznetsov

[cblecker]

@nickvanw It doesn't seem that the v3 API provides a way to add a card to a board in a "pending" state.. you need to add a card directly to a column. The v4 API notes that a card can be in a pending state, but again, it requires you specify a column manually.

If we were able to add in a pending state to a board (as the web UI does) then the automation would take over and put it in the right column per the settings.

[taragu]

@cblecker Thanks! This is very helpful. Do you know if/when we are planning to switch to using v4?

[taragu]

@jlewi so looks like with the Github API v3, we are not able to use the project automation to add issues/PRs to a default column. The workaround is to set the default columns for each projects. I have a couple of options for users to set the configs for default columns. Which one do you prefer?

• Set default column for each project in plugins.yaml; if no default column is specified, user has to specify the column in the command (/project 0.4.0 To do) or the command is invalid.
• Set the default columns for each org and repo in plugins.yaml: if we are adding an issue to a project, and one or more of the default columns exists for the project, then the issue will be added to the first match in the list of default columns. If no default column is found for a project, the command is invalid.

[stevekuznetsov]

I feel like the pattern that would most likely match the workflow for an org would be to make defaults based on org/repo, since I'd assume people that make N projects will have similar defaults between them, but that's all conjecture. Let's choose something that makes k8s happy for now and wait for v3 API to support the best workflow?

WDYT @cblecker

[cblecker]

There's no real rhyme or reason to how columns are named.. I'm inclined to go with option 1 IMO.

So:

• If a valid column is specified in the command, use that.
• No column is specified, look for a default for that project in plugins.yaml. If that exists, use that.
• Comment with the failure, and suggest a column be specified. Extra points if we can list the columns (nice to have, but not required).

[taragu]

Sounds good @cblecker Thanks for your input!

[taragu]

@jlewi @stevekuznetsov @cblecker fyi I've updated the config to the following. Please let me know if you have any other suggestions.

project_config:
  project_org_configs:
    kubernetes:
      org_maintainers_team_id: 2460384
      org_maintainers_team_name: kubernetes-milestone-maintainers
      org_default_column_map:
        1.14 CI Signal:
          Start here
        KEP Implementation Tracking:
          To do
      project_repo_configs:
        kubernetes:
          repo_maintainers_team_id: 2460384
          repo_maintainers_team_name: kubernetes-milestone-maintainers
          repo_default_column_map:
            component-base:
              To do
            Workloads:
              Backlog
    kubeflow:
      org_maintainers_team_id: 2460384
      org_maintainers_team_name: kubernetes-milestone-maintainers

[jlewi]

This is great thank you so much for doing this!

[taragu]

@jlewi you are welcome! This is still currently being tested. Will let you know when this is ready to be rolled out.