Community: PR Lifecycle stats, automation, and leadership notification

[timothysc]

I decided to start to investigate why do we have so many open PRs and how long have they been open for, and the data is troubling IMO.

is:pr is:open created:<=2019-04-07= 153
is:pr is:open created:<=2019-07-07= 432
is:pr is:open created:<=2019-09-07= 747
Total open PRs = 1053

What are we doing to ensure PRs are reviewed in a timely manner and that SIG leadership is aware of these numbers.

/assign @castrojo
/sig contributor-experience
/cc @kubernetes/steering-committee

[timothysc]

Also worth noting is:issue is:open updated:<=2018-10-07 = 202 .
Meaning there are 202 issues open that haven't been updated in over a year!!

[geekygirldawn]

/assign

I'd be happy to help @castrojo or others dig into this issue to see what we can do to help better manage our PRs.

[cblecker]

Right now we close untouched issues after 150 days:

• 90 days until issue goes stale
• Another 30 and the issue goes "rotten" (basically, gets a second reminder)
• Another 30 days after that and it gets closed

For the issues where this is not happening, it's because they have been "frozen" to prevent interactions from the lifecycle bot. For example, kops has 92 frozen issues:
https://github.com/kubernetes/kops/labels/lifecycle%2Ffrozen

[alejandrox1]

/cc

[mrbobbytables]

Already digging in to this from the owners perspective.

/assign

[timothysc]

Any action plan here I'd like to see a policy doc written on lifecycle of an ISSUE + PR in order to help:

1- Accurate routing
2- Timely response
3- Appropriate remediation
4- Overall expiration

I recognize some issues are thorny and can take a long time to address, but there is little reason a PR should languish for 6+ months, and it should be a rare exception.

[castrojo]

From a quick call with Tim, it'd be neat if the lifecycle doc was a spec for the bot to do comments on PRs and issues based on the lifecycle, for example "It's been 15 days since this PR was touched, please see step foo in the lifecycle document and contact baz." as an example.

[timothysc]

Hey folks what priority is contribex giving this?

[mrbobbytables]

I can't speak for others, but it's on my list. I'm a little swamped right now between work and KubeCon/Contributor Summit prep though 😬

[parispittman]

@timothysc we will discuss in the meeting today but I have a feeling most folks that work contribex are working the summit or one of the 400 kubecon events. Could we attempt to do this as an unconference at the summit to talk about PR hygiene?

Do we know what SIGs are labeled on those PRs? We can start with a spreadsheet and then I can work with SIG Chairs who have n days old things that need to be triaged. I've also included a 'please build a triage team' on latest chair comms. We should probably template out a team (with rolebooks) so folks can apply quickly and everyone isn't reinventing the wheel.

[celanthe]

@parispittman I'm happy to help with setting this up as an unconference event at the Summit, could be great way for those who are interested in getting involved with the project but aren't sure where to start/how to contribute. :) If there's a point of contact I can get in touch with to secure a space for it/someone I can talk to to get it on the contributor site schedule I'll make it happen!

[alejandrox1]

helping out on this as well so let me make sure i don't lose track of this
/assign
On another note, per @parispittman 's request. I made a simple program to see what labels appear in the PRs that have been open the longest (alejandrox1/cool-github/pr-stats in case you anyone needs it).
I ran this program to see the labels on all PRs that are open and were created more than 3 months ago and came up with this...

- 408   cncf-cla: yes
- 257   release-note-none
- 228   needs-priority
- 183   ok-to-test
- 149   kind/cleanup
- 130   release-note
- 126   kind/bug
- 111   size/XS
- 110   sig/node
- 96    sig/api-machinery
- 94    area/kubelet
- 94    lifecycle/stale
- 92    size/L
- 91    size/M
- 82    kind/feature
- 80    sig/testing
- 79    size/S
- 76    sig/cli
- 73    area/test
- 73    lgtm
- 72    needs-rebase
- 70    area/kubectl
- 70    sig/apps
- 68    lifecycle/rotten
- 65    priority/backlog
- 56    needs-ok-to-test
- 56    priority/important-soon
- 53    do-not-merge/hold
- 46    sig/cluster-lifecycle
- 44    sig/storage
- 43    kind/api-change
- 38    priority/important-longterm
- 37    area/apiserver
- 34    sig/scheduling
- 32    sig/network
- 30    approved
- 29    needs-kind
- 28    do-not-merge/work-in-progress
- 27    do-not-merge/release-note-label-needed
- 25    size/XL
- 20    sig/cloud-provider
- 18    sig/auth
- 17    kind/documentation
- 15    area/dependency
- 14    area/kubeadm
- 14    size/XXL
- 13    lifecycle/frozen
- 11    needs-sig
- 11    sig/architecture
- 9     area/cloudprovider
- 9     sig/instrumentation
- 9     sig/release
- 9     sig/windows
- 8     area/code-generation
- 7     cncf-cla: no
- 6     area/provider/aws
- 6     area/provider/gcp
- 6     area/release-eng
- 5     api-review
- 5     area/conformance
- 5     area/ipvs
- 5     sig/autoscaling
- 5     sig/scalability
- 4     kind/design
- 4     kind/failing-test
- 3     area/e2e-test-framework
- 3     do-not-merge/contains-merge-commits
- 3     kind/flake
- 2     area/provider/vmware
- 2     do-not-merge/cherry-pick-not-approved
- 2     wg/apply
- 1     area/app-lifecycle
- 1     area/client-libraries
- 1     area/code-organization
- 1     area/kubelet-api
- 1     area/provider/azure
- 1     do-not-merge
- 1     do-not-merge/invalid-commit-message
- 1     priority/awaiting-more-evidence
- 1     priority/critical-urgent
- 1     release-note-action-required
- 1     wg/component-standard
- 1     ¯\_(ツ)_/¯

[alejandrox1]

There are some other things to "explore", i.e., what labels appear tend to appear together for the old PRs and then to repeat this on all PRs that have been closed due to a lifecycle rotten label.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[mrbobbytables]

/remove-lifecycle stale

[mrbobbytables]

@matthyx has actually done work in this space lately :)
Here is a google sheet with some very useful stats (created_at, closed_at, sigs etc):
https://docs.google.com/spreadsheets/d/1umeNRqS5Mqn53LjKFfH4S5I31qaVLcCND_2PHu5UkPE/edit?usp=sharing

He's made his scripts available here:
https://gitlab.com/matthyx/repo-stats

[matthyx]

Hi, I have added some more graphs, but they aren't too useful as such...
Can any of you suggest some edits or new data?
We can also discuss in Amsterdam in one month!

[LappleApple]

Hi all: this has been moving forward, but has lots of moving parts/SIG specifics to wrap one's head around so it takes some time. Help is welcome, mainly around SIG outreach; get in touch if you'd like to contribute on that front.

Meantime, wanted to share this DevStats feature request—it aims to automate a manual data collection effort that I conducted in April to better understand the PR velocity issue (+ general workflow and triaging processes in SIGs). If you want to see the results of that effort, please ping me on Slack (name: Lauri Apple) and I'll share my Google Spreadsheet.

[LappleApple]

/assign

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[matthyx]

/remove-lifecycle stale

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale

[LappleApple]

/remove-lifecycle stale -- still actively working on this via different angles

[mrbobbytables]

/remove-lifecycle stale