Community: Kubernetes Elders Council - proposal

[sarahnovotny]

Problem: As the Kubernetes project has pivoted from a Company run open source project to a Foundation owned and community inclusive project, there have been growing pains. The perceptions of Googler who have more structure and visibility into most of the operational aspects of the projects differ greatly from the perspectives of most of the community members - particularly new project contributors and interested project contributors. Google intentionally chose not to have a BDFL for all the down sides that has, but a confused “power vacuum” has evolved. The organic structure appears to a newcomer to be opaque, relationship based and full of cronyism. There is no clear starting point progression in the project or escalation for conflict resolution etc. Long standing (primarily Google employed) contributors feel that newcomers are all focused on specific features and aren’t stepping up to contribute to the more material and growing backlog of housekeeping, management, and organizational health tasks.

Goals : Establish clear cultural, technical, and team leadership as well as act as an escalation point with a “last say” on any contentious issues.

Requirements: This group should have a long standing history with the project, good people management skills, strong technical vision for the project and empathy for the user and FOSS developer perspective.

Instantiation: The first seed incarnation of the Elders will be voted in as a slate of 3 Elders with a 2 year term (ending August 2018) through the Community Meeting and on the mailing list kubernetes-dev@. During the first year, the Elders will add 2 members (with a term ending August 2017) to their group for a total of 5 members.

Refreshes: In August each year subsequent the community will vote on the seats which term out. All positions voted on will have a 2 year term. The voting will be handled using Condorcet voting and a service such as CIVS. Voter eligibility must be defined, and will require registration during a public registration period. Nominations for candidates will be accepted for one week prior to the voting period.

Resignations: In the event that an elder resigns, the remaining elders will appoint a successor to serve until the next scheduled vote.

Known Issues: Elder nomination process. Voter eligibility.

[timothysc]

sounds like apache-"PMC" + with rotation.

[dims]

Or the OpenStack Technical Committee:
https://wiki.openstack.org/wiki/Governance/Foundation/TechnicalCommittee

[erictune]

Is there a forum or issue yet to discuss voter eligibility?

[chrislovecnm]

Goals are listed as:

Establish clear cultural, technical, and team leadership as well as act as an escalation point with a “last say” on any contentious issues.

To me, those goals are kinda vague. What are some example duties? What influence does the governing body have?

Moreover, what checks and balances are in place? How can the council be dissolved in case of serious leadership issues? How do we keep politics to a minimum? How can we make being elected to the council, not a popularity contest? Would we be able to have 1-year term?

[philips]

@sarahnovotny thanks for putting this together. Seems like a great start and we do need to sort something out as our alternative to BFDL.

One anti-pattern I want to avoid is the "detached leader" who shows up just during critical moments but doesn't have the confidence of the community as they aren't regularly engaged. A few ideas on how we might do that:

1. Ask elders to rotate through planning and managing a major release. This likely looks like the role that @mike-saparov and @idvoretskyi took in the v1.3. Essentially, report to the community hangout feature progress, keep tabs on super critical issues, and do all they can to keep velocity up on the 3 or so super critical items for that release.
2. Ensure that elders remain engaged in the community through Kubernetes community projects, code, mailing list participation, or running of community meetings. By Kubernetes community projects I mean really wide stuff from kubernetes.git to stuff like @brendandburns ksql.
3. Make themselves available through office hours on a twice monthly basis to discuss issues with contributors and maintainers and try and work through tricky core issues in real time.

[countspongebob]

Sarah, much appreciated effort and really critical to the health of the project, thank you!

Moving the project to a state where the power structures are transparent is vital.

The two-year terms with offsetting terms for part of the group is a good term with enough time for context and continuity.

I entirely support Brandon's comments about ensuring that we don't have detached leaders, and it seems pretty likely that the workload for this position will not be minimal. We should keep the council of elders separate from the CNCF TOC (you can serve on either CNCF TOC or Elder council) as some level of interaction between those two bodies is likely, and workload is an issue. I also propose that although SIG leadership is a good eligibility indication that Elders should recuse themselves from running a SIG.

The Openstack TC is far too large... I think the board sizing you propose is right.

[mattfarina]

Sarah, thanks for putting this together.

While I'm not a long timer I like this idea very much. The small size and odd number are ideal as well.

In the first setup of terms can we try for at least one non-Googler to be an elder? I think that would be important for the community.

I second all of Bobs comments.

[josephjacks]

@sarahnovotny Thanks for this. I concur with _most_ of what you wrote in the problem statement.

I think @philips makes some solid points about how we can avoid "disengaged/detached" leaders. I'd like to add related thoughts around the elder nomination/eligibility criteria along with other ideas/concerns generally:

• In addition to their Elder status and duties, Elders should not be materially leading/directing any commercialization efforts around K8s. This would hopefully help mitigate any bias that could creep in and taint decisioning/oversight.
• We need much more clarity in Goals. Can we unpack "contentious issues" so everyone is super clear here?
• We should better define "long standing history with the project". Does this mean at least since v0.5? v0.4? I think this matters as much as weight and depth of contribution relative to time period.
• Is this happening by "executive decision" within Google or is the OSS community of the project going to determine if this proposal gets implemented? I'm sure we will see a lot of feedback on this.

[chrislovecnm]

We need much more clarity in Goals. Can we unpack "contentious issues" so everyone is super clear here?

@sarahnovotny and others, it would be good to have a working group on defining mission and goal of this Elder Council.

We should better define "long standing history with the project". Does this mean at least since v0.5? v0.4? I think this matters as much as weight and depth of contribution relative to time period.

Agreed

[bprashanth]

@jfrazelle fyi

[jessfraz]

This is really neat!!! I have a bunch of people I think would be perfect!

[sarahnovotny]

Thanks for all the comments.

As fodder for the first charter iteration, please comment with examples of things that you'd imagine being escalated to the elders council. They can be concerns from the past, current issues or hypothetical cases. Please also include the type of resolution that you might expect ... not the outcome, but types like a technical decision, a policy, a process etc.

[erictune]

Examples of conversations I think we might have, that we _might_ escalate to the elders:

1. Does workflow belong in kubernetes/kubernetes, or in another project? Hypothetically, if the relevant SIG had said no, the advocate for adding workflow might escalate to the elders. The elders would make a decision such as "yes, now", or "no, never", or "not at this time". Their answer would probably include reasons why they decided one way or another.
2. Someone wants to add a webhook admission controller which sends every request to a webhook. Agreement cannot be reached on whether this is a good idea.
3. Sig-Apps leads want to add Sticky IPs. Sig-Network leads say that sitcky IPs are too hard to implement. Someone does an implementation that works on 1 cloud provider. Sig-Network leads say that it won't work with some other types of networks. They escalate to the elders for a decision.
4. One group wants to add an "image build resource". Another group says this is too "opinionated" for Kubernetes. They cannot agree. Elders decide, giving reasons.
5. One group wants to do something that seems like a breaking change to an API but another groups says it is not a breaking change. Elder decides whether it is.
6. Someone wants to add dynamic loading of c libraries as a new form of plugin. Someone else says this is too hard to maintain in the long term. Elders decide.
7. Someone wants to make heapster an effectively required part of kubernetes. Someone else, who works for a third-party monitoring provider, wants their solution to be able to plug in. Elder could decide.
8. A storage vendor really needs changes to the storage plugin interface to enable their cool feature. The OWNER of the relevant code (e.g. storage controller) disagrees, saying this makes the code much harder to maintain and understand. Vendor is very upset, since her business is on the line. She feels the OWNER is being parochial. She escalates to the Elder, who is widely known for fairness.

[erictune]

Common themes for these examples:

• There is always a single technical issue at stake. There may be hurt feelings and history and baggage involved, but the question and decision are about a technical issue.
• Discussion occurred with OWNERs or SIG leads first, and got stuck.
• The lines are already drawn before it reaches the Elders.
• Someone who is not a SIG lead or OWNER can request an escalation to elders.
• Questions often center around "I want this feature" vs "this would hurt the project in the long term".

Therefore, Elders should be people with deep technical expertise, and experience in large, long-lived software projects, and a long-term commitment to this project. Those attributes should, in my view, trump the other requirements.

[Lukenickerson]

I'm curious about the difference between the responsibilities of the Elders versus the existing SIG leads. Are SIG leads already tasked with providing leadership, making decisions on what features get included, and resolving contentious issues? If so it seems like the Elders would act as the next-higher level of leadership and judgement, providing leadership _across_ the SIGs and resolving (hopefully rare) issues that cannot be resolved by the SIG leads. Is that right?

[shtatfeld]

I really liked @erictune escalations to the elders examples as they focus on technical issues and also they define the relationship between elders and SIG leads.

In the original proposal we are trying to address 2 issues, and perhaps the elders is a solution just to one of them:

1. Escalation point for technical matters - Elders as @erictune described can address.
2. Processes, roles and responsibilities - we have started with the contribx working group, focusing mostly on tools but we can expand it to processes too, like the roadmap suggests and define set of roles in the community (similar to SIG leads, e.g. release czar), or maybe form a new working group to focus on the community operation. Escalation point on this matters can be to @sarahnovotny. This team can also address how we ensure transparency (e.g. sig's meeting notes, org chart?).

@aronchick we should also add the community pm team as one of the sigs and apply the same practices we have (or trying to establish) for sigs.

[eparis]

I have to put a +1 vote in for @erictune idea of an elders group responsible for escalation of technical decisions.

[idvoretskyi]

@shtatfeld +1 to add community pm team as the SIG. Let's work on this /cc @philips @aronchick @sarahnovotny

[sarahnovotny]

Kubernetes Elders Council DRAFT v2
Define the Kubernetes Elders program, responsibilities, vision

Elders Council :
This group acts in the service of the community and the project to provide clear decisions and direction for the community when it is unable to independently make a decision. Whereas most of the component level technical decisions happen in Special Interest Groups (SIGs), the Elders are expected to steward a broader Kubernetes Project vision through clear resolutions in situations like multiple SIGs having conflicting views or larger community discussions which don’t have clear outcomes. Additional examples are cited comments in the initial community discussion.

Escalation: Publishing a process to escalate to the Elders will be one of the first tasks of the group in conjunction with Sarah Novotny. A recommended process is an Elders escalation mailing list where discussions can happen between representatives of opposing positions on an issue. Decisions documented in kubernetes/community and a mail to Kubernetes-dev@ mailing list.

Status: This process is light on legalese, completely untested, and only works if people act as good neighbors and community members. It will evolve over time and the authors will try keeping the process light, fast, and objective.

Requirements: Members of group will be active and have leadership positions in the community as well as have a long standing history with and future commitment to this project in addition to experience in large, long-lived software projects. Key to success will be people management skills, strong technical vision for the project and empathy for the user and FOSS developer perspective.

Instantiation: The first seed incarnation of the Elders will be voted in as a slate of 3 Elders with a 2 year term (ending August 2018) through the Community Meeting and on the mailing list kubernetes-dev@. During the first year, the Elders may add 2 members (with a term ending August 2017) to their group for a total of 5 members.

Refreshes: In August each year subsequent the community will vote on the seats which term out. All positions voted on will have a 2 year term. The voting will be handled using Condorcet voting and a service such as CIVS. The vote will not be public, but will require standing as an Active Contributor as defined in the Community Expectations (currently being discussed in - https://github.com/kubernetes/kubernetes/pull/32096 ). Nominations for candidates will be accepted for one week prior to the voting period incumbents will be automatically nominated unless they opt-out of reelection.

Resignations: In the event that an elder resigns, the remaining elders will appoint a successor to serve until the next scheduled vote for that seat.

[philips]

LGTM

[shtatfeld]

LGTM
Thanks Sarah!

[marun]

What do people think of adding a term limit to the proposal? My experience in OpenStack suggests that influence is extremely sticky. In any given election, incumbents almost always prevailed against challengers. This reduced the incentive for incumbents to be accountable, and made it very difficult to broaden the base of leadership in the community.

[erictune]

What limit would you suggest?

[marun]

Given a 2 year term, I think it would make sense to prohibit consecutive terms. I don't think there should be a limit on serving non-consecutive terms.

[erictune]

I'm not sure I can think of 10 people right now I would want to be elders. I guess either I am too picky, or it would be a gamble that we can grow more elders in the next 2 years.

[marun]

That begs the question: How would someone qualify to stand for election?

I think a certain amount of unease with relative inexperience is to be expected. Always preferring the known quantity, though, is not without cost in an open source community. In the absence of a benevolent dictator, responsibility for the long-term sustainability of the project would ideally be distributed to as many capable individuals as possible.

[idvoretskyi]

@sarahnovotny LGTM.

I'd like to suggest to place this proposal into a separate document - it will be easier to review, discuss and update it there (leaving the general discussion in this thread).

[eparis]

@erictune it's only 8 not 10 if they take 1 year off. But I don't think that's actually relevant. I'm not aware of all of the issues around OpenStack, but it seems that continuity of thought leadership would be a good thing. Does that fact that _fingers crossed_ the elders will have little work to do as the SIG leaders will be the deciders of most thing reduce the concerns at all?

[marun]

An elders council may function as an arbiter of last resort, but as per @eparis it will not be the only source of wisdom and experience the community has to draw upon. One person's election as a thought leader doesn't preclude others from attaining or retaining thought leadership.

[chrislovecnm]

Any movement on this?

[davidopp]

-1 for term limits

[jbeda]

Do we want to have any sort of limit about how many people from a specific company can be involved? We are going to want to build diverse community and it would be unfortunate that the project leadership be dominated by any one company.

I'd propose that any one company can only have one position voted in (at the time of the vote) and at most 2 members on the council at any one time.

(The position follows the individual, of course. And people move around so we wouldn't kick someone off for changing jobs. But we could validate this invariant at vote time.)

[philips]

@jbeda the two at a time limit seems reasonable; although I don't know if the number voted in at a time is as useful.

[idvoretskyi]

@jbeda agree with @philips. The question is - what will happen if more than one person from some company will move to another company, that has already delegated some people?

[jbeda]

I would suggest that we don't cut anyone's term short based on where they work. That doesn't seem right. Instead we say something like "Any newly elected member should not increase representation on the board above 2 people per company. In the case where this invariant would be violated, the candidate(s) with the highest number of votes will be elected."

So say we have persons A, B, and C form company X and person D from company Y and 3 open spots. They could all appear on the ballot. But if the results are (in order) [A, B, C, D] we would disqualify C (because it would violate our invariant) and take [A, B, D] to fill the slots.

If someone gets hired and increases representation above the "2 people" rule nothing changes and we will have a company (temporarily) overrepresented on the board. At the next election we'd look to fix that and restore balance.

[chrislovecnm]

@jbeda I like that!

[countspongebob]

I have a related (and supporting) proposal to offer: https://github.com/kubernetes/community/issues/295

[bgrant0607]

Superseded by Steering Committee:
https://docs.google.com/document/d/1btvNcAKHG-3vxxpnoOk0dV5-RqyYyd198Psg11cEPVA/edit