Test-infra: Explicitly invite users to join the org after several merged PRs

[Katharine]

What would you like to be added:
It would be nice to have trigger include a bold, explicit invitation to join the org in its ok-to-test message after you've made a few PRs (three? five?), perhaps something vaguely like this, totally off the top of my head:

We noticed you've done this a few times! Consider joining the Kubernetes org to skip this step, as well as gain LGTM and other bot rights rights. We recommend asking the approvers on your prior PRs to be your sponsors.

(It would also be nice if we had a document to link to that wasn't all about reasons you can't be a member...)

We can do a query like type:pr is:merged org:kubernetes author:$user before making the comment to determine if someone is qualified - this graphql query has a flat cost of 1:

query {
  search(query: "type:pr is:merged org:kubernetes author:katharine", type: ISSUE) {
    issueCount
  }
}

Alternatively, we can just always comment and leave the user to figure out whether they count, but I think this diminishes the effectiveness over a positive assertion.

Why is this needed:
Joining the org is pretty intimidating. On top of that, it is wildly unclear when people are eligible to join, and even if they clearly are, it often doesn't occur to them to actually do so. While this doesn't help contributors who do not make PRs who would be eligible for org membership, it would be helpful to those who do. The real bar is much lower than non-members tend to think it is.

We currently mention that "regular contributors should join the org to skip this step", but it's not at all noticeable and "regular contributors" is wildly open to interpretation.

(shoutout to @liztio for the concept)

[cblecker]

It's not about number of PRs though. It's about the contributions you bring to the project.

50 spelling fix PRs isn't the same as 2-3 quality contributions.. or zero code contributions but regularly attending meetings, providing feedback, or taking on non-code tasks.

I recognize that imposter syndrome is a problem, but I don't believe it's one that bots can solve. I personally think it's more about the people involved.. including having reviewers/approvers actively reach out to folks they think qualify.

[liztio]

I don't think we're going to solve imposter syndrome entirely, but I don't think there's a lot of downsides to implementing this. If we end up getting completely flooded with unqualified people trying to join the org that's one thing, but I really doubt that's going to happen.

And relying on people to "notice" others' contributions is unlikely to be a good solution. How would we go about publishing such an initiative? We already know what we have now isn't working.

[Katharine]

@cblecker: that applies just as much to our "regular contributors should join" message that we already give on every PR that a non-member makes. This just makes the existing suggestion more positive and less ambiguous. It doesn't auto-approve membership, it just suggests that you should consider applying. It's not attempting to solve all angles of the problem, just be more encouraging to some subset of likely qualified users with whom we happen to have a channel available.

I think it very unlikely that this is unlikely to have any negative consequences.

[markyjackson-taulia]

I love this idea

[coderanger]

This is delightful, even if we have to bikeshed about thresholds :)

[spiffxp]

/sig contributor-experience

My only nit is this doesn't recognize those folks who do great work that doesn't involve contributing code. However, measuring that is... more involved. Agree this is a step in the right direction.

Linking to the issue I've used as umbrella for all things OWNERS which I consider proxy for measuring/enforcing things related to contributor ladder https://github.com/kubernetes/community/issues/1808

[cblecker]

Making language more friendly (in general): 👍
This applies to both the bot message on the PR, as well as language improvements to the community-membership.md doc.

Still not convinced though that tying it to number of PRs is a pattern we want to use. We previously had PR numbers in the community membership document, and it just became a "goal" to hit. Not entirely closed to it, but I don't want to trade one negative behaviour (folks who are qualified not applying) for another.

As I bat this around in my head maybe something like the following:

• X number of PRs, but higher than the "low" bar (15 for example)
• And/Or a time since first PR search term (don't know if this would make it better or worse)
• Don't publicize or refer to the number of PRs in the message. Some extra bolding or more suggestive messaging instead.

If we frame this more this to catch outliers.. those folks who are probably over qualified, but haven't applied yet.

It's just avoiding some number being the goal. I also don't want this to be a substitute for what should actually be happening, which is reviewers/approvers seeking folks to join the org (or for promotion to reviewer/approver, etc).

[spiffxp]

Some prior discussions when this has come up in the past

• https://github.com/kubernetes/org/issues/10
• https://github.com/kubernetes/community/issues/874

[nikhita]

Some ideas here, very much open to dissenting opinions.

I think the root cause is that our membership requirements are ambiguous and the "bar" is not defined. If I'm applying for membership, I am not sure if I'm good enough, because I don't know what "bar" I'm supposed to be looking at. This has also led to the bar being inconsistently followed in different membership requests, which would make me even more confused if I was applying today.

We previously had PR numbers in the community membership document, and it just became a "goal" to hit.

I agree that looking at just the number of PRs isn't helpful because it's possible that all PRs are trivial (typo fixes, etc). I think what we are looking for is _significant_ contributions. "Significant contributions" don't necessarily have to be PRs either.

IMHO one solution would be to:

1. Concretely define how many significant contributions we expect, _if these contributions are PRs_ (maybe 5/10/15?). If they are not PRs, the requester is free to describe their contributions as they'd like. This is a little similar to what we did for the last steering committee elections, but not centred around devstats (which can be easily gamed). Having a goal to hit 10 significant PRs is IMO not a bad thing. :)

2. The decision of whether a contribution is significant enough would be left to the sponsor since they would be an approver/reviewer for the relevant subproject.

While not having concrete numbers around PRs makes sense, I think we should define some number around significant contributions, _if they are PRs_ because that helps set some bar. At the same time, it also allows people to describe their non-code contributions.

This is also a little similar to what we have today -- a membership request requires a list of contributions (if this is PRs, we already expect these contributions to be non-trivial) and the request is good to go if the sponsors are cool with it. IMO the minimum number of significant PRs limitation just adds some structure to the current (loose) definition.

Additionally, we could also add an FAQ-like section in our membership document. Kind of reiterates what's already in the requirements but makes it less intimidating.

# Applying for membership

## Should I apply for membership?

If you have made at least 10 significant PRs _or_ any other significant contributions and plan to remain an active contributor, you should seriously consider applying for membership!

## How do I find sponsors?

Ask the reviewers/approvers on your PRs, or people that you have interacted with on issues, slack or meetings.

## How do I know if my contributions are significant enough?

Talk to your sponsors about it! Describe your contributions and ask them if they would be willing to sponsor you. You could also ask them about the kind of contributions that they'd expect for membership and what you can do to climb up the ladder.

Instead of adding a message on PRs when the author has >= X number of PRs, we could add the message to the needs-ok-to-test message on _all_ PRs.

If you've made multiple contributions, consider joining the Kubernetes org to skip this step, as well as gain LGTM and other bot rights rights. We recommend asking the reviewers and approvers on your prior PRs to be your sponsors.

Wdyt?

[markyjackson-taulia]

I agree with you @nikhita but 1 question I have is around this: If you have made at least 10 significant PRs: What is the criteria for significant PRs?
I feel that getting users to contribute in any fashion is a step in the right direction and even a doc fix is significant. Perhaps there can be levels or membership or mentorship to becoming an member. I have found that making people feel they are doing in any fashion will lead to greater things and we should be inviting more people.
Speaking from my own experience, I have found that becoming a member is much harder then expected and some SIG's for example are not very welcoming and this leads to disappointment. This is of course my opinion and outside of this discussion.

[nikhita]

What is the criteria for significant PRs?
I feel that getting users to contribute in any fashion is a step in the right direction and even a doc fix is significant.

Significant = anything that would make reviewers/approvers go "I think the requester has made some non-trivial contributions and are an active contributor. Having them be a part of the org would be helpful because that'll allow them to be assigned to issues and lgtm PRs, eventually moving to become a reviewer".

The decision to whether a PR is significant enough or not would be left to the sponsor.

Doc fixes are definitely non-trivial! :+1:

Come to think of it, I think calling it "significant" might be somewhat discouraging. Maybe just go with non-trivial? Not entirely sure, will need to noodle on this a bit more. :)

Speaking from my own experience, I have found that becoming a member is much harder then expected and some SIG's for example are not very welcoming and this leads to disappointment.

:disappointed: happy to talk this over in #sig-contribex and look into improving this.

[markyjackson-taulia]

Thank you for the response @nikhita it is truly appreciated.

[Katharine]

Instead of adding a message on PRs when the author has >= X number of PRs, we could add the message to the needs-ok-to-test message on all PRs.

If you've made multiple contributions, consider joining the Kubernetes org to skip this step, as well as gain LGTM and other bot rights rights. We recommend asking the reviewers and approvers on your prior PRs to be your sponsors.

Wdyt?

We already have a message that says half of this. Rewording it is an option, but I don't think it is likely to help because nobody will read it in all the piles of bot noise that shows up on every PR. Especially since, by definition, we're trying to catch people who have seen this message many times and will not read it at all unless something is very obviously different.

As mentioned in the original issue, I prefer the filtering option because it lets us be more assertive - we might not know with certainty whether the contributions were "significant" (whatever that means), but if we can pick a point to _change_ the message and include a bold instruction, I think that's a significantly more compelling comment than slightly adjusting a message that's always there.

---

As to the rest of the comment: I'm very in favour of having a positively worded document (or document section) that says when you should apply and explains how to get through the nuances of the system (e.g. finding sponsors). I don't think the existence or absence of that is directly in scope for this issue, though.

(I also think some of our existing requirements just don't accomplish anything and unnecessarily put people off, but that's _way_ out of scope for this.)

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[michelle192837]

/remove-lifecycle-stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[spiffxp]

/remove-lifecycle rotten

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[michelle192837]

/remove-lifecycle stale

I still think this would be a good, inclusive improvement.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[michelle192837]

/reopen

[k8s-ci-robot]

@michelle192837: Reopened this issue.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.