Wpt: Some sort of governance/process

[gsnedders]

This is something that's mostly been discussed on IRC at various points.

There's a number of issues here:

• Where do policy decisions get made? (The status quo appears to be all over the place.)
• How do we ensure interested parties are aware of where the decisions are being made? (The status quo appears to be random subsets of people.)
• Is there anyone to whom you can appeal the decision?

cc/ @web-platform-tests/admins @annevk @domenic (who I think are the two who have brought this up most often?)

[foolip]

Where do policy decisions get made?

How do we ensure interested parties are aware of where the decisions are being made?

For both of these I would suggest a new repo, called "policy", "governance" or "meta", which is low-volume enough that it's possible to subscribe to it.

Is there anyone to whom you can appeal the decision?

@zcorpan and I discussed this a bit today. A steering group in the style of https://github.com/whatwg/sg might work, with the same expectation that the mechanism will always never be used, because people generally prefer to resolve their issues without escalation, where they have more control over the outcome.

The tricky part is how to create the initial group. Because it should be "boring" and effectively never used, having a big election or similar would make it seem more important than it is. My best suggestion might be to use the same procedure as when picking the logo, namely asking the top 10 wpt contributors vote if there are too many candidates.

[foolip]

@jgraham recommended that I watch How Rust Does Open Development. It's a long talk, but 2m12s is a brief history, 6m8s has the talk outline, and 46m1s has the main lessons.

The answer to WWIC? was an RFC process for "substantial" changes. Documented at https://github.com/rust-lang/rfcs. Not clear if there's a fast path when everybody agrees, I expect that'd be the case in WPT 90% of the time. I think a process like this would be good, a single place to monitor for "substantial" changes rather than relying on others poking you on big issues/PRs.

For a process like that to work we would still need a way to resolve disputes, i.e. a group of people who are trusted to make reasonable decisions. @web-platform-tests/admins could be this group, but it was created with no transparency, and I agree with @annevk (on IRC) that it's probably best to not conflate the two roles. Instead, we could create many teams or working groups as in Rust and have them decide in their areas, or perhaps have a single steering group intended as an infrequent escalation path.

There's also a lot of discussion on how to attract new contributors and help them advance to regular contributor and roles of leadership. This is an area we're frankly not doing very well in. This Week in Rust looks nice, I'd love a "This Week in WPT".

On planning larger arcs of work, I like the "slogans" idea, where the goals for a whole year are condensed into single short slogans. Currently we have no shared roadmap at all for WPT, only things like Ecosystem Infra 2018 OKRs which represent just what one company is up to.

[jgraham]

After watching that talk and thinking abou the different requirements of wpt compared to Rust, I formulated a proposal. I'm not sure it's the right end point, but it might be a useful starting point for discussion. It notably doesn't cover people management issues like who enforces the CoC and how.

Goals

Have a system of governance for the project that

• Offers cross-organisation leadership over project direction and priorities

• Allows the community to follow the wpt work, and have input into the
  overall direction and into specific technical questions.

• Provides a clear escalation path and decision making process both
  for technical and project management issues.

Influences

• Heavily influenced by the Rust
  project
  which in turn takes inspiration from TC39 amongst others.

Proposal

• Nominate a "core team" who are responsible for final decsion making
  and organisation of the project. The team should consist of ~5
  people(?) representing at least 3 different browser engines. The
  process for selecting/replacing core team members is by agreement;
  not by voting.

• The core team may appoint further subteams as requied (for example
  one could imagine a team responsible for wpt.fyi or infrastructure
  or whatever). Each subteam should contain at least one member of the
  core team.

• The core team is responsible for

  • Setting and communicating the overall priorities and direction
    of the project

  • Taking overall responsibility for shared infrastructure (GitHub,
    any shared password, etc.)

  • Being final arbiter on technical decisions

  • Making changes to the project goverance.

• The core team may delegate day-to-day running of any part of the
  project to subteams.

• In order to set and communicate priorities the core team commits to
  publishing a blog post periodically, setting out the priorities and
  direction of the project for the following period.

Making Changes

User-facing changes to the project require an issue in a special
repository that will form a "RFC". Examples of changes that might
require such an issue include:

• Adding a new test-writign API to wptserve

• Adding new features to testharness.js

• Adding new test types or adjusting how tests are evaluated
  (e.g. adding fuzzy reftest matching).

  There are several reasons to require an RFC-type issue in these cases:

• Allows people to follow substantive changes to the project, in a
  setting that is much lower volume than the main wpt issue tracker.

• Ensures that changes can meet the needs of all participants in the
  project before they are rolled out.

  It is highly desirable that the process remain lightweight; compared
  to a programming language decisions made for wpt are generally much
  easier to reverse. Therefore in the case of no substantive
  disagreement changes may be assumed to go ahead after a reasonable
  period (e.g. 1 week).

  In the case that an issue generates substantive discussion it is
  assumed that the proposal will change in accordance with that
  discussion. If participants are unable to agree on the correct
  approach for a change — or whether the change may be made at all —
  then the core team (or relevant subteam) must make a decison. The
  exact mechanics for this are unspecified, but they must only consider
  arguments that have already been made on the RFC issue.

[plehegar]

I like the direction. Some comments below:

I would to make the "core team" to be at least 6 if we're keeping a requirement to have at least 3 browser engines. With only 5 and the requirement to have 3 browser engines, you're preventing the project to be independent from browser engines, which was always important in my mind to have on the paper, even if the reality is different (due to current major contributors). Having the potential to have some independence is important imho.

The scope of WPT, as a project, shouldn't be restricted to "unit tests for browser engines" imho. The overall project is about moving the Web forward using tests. I grant you that the current WPT repo achieves a lot for that. But I could imagine for example a separate repo called web-platform-tests/wpt-app-tests, which would focus on Web applications testing (like specific UI scrolling pattern/practices). So, those wouldn't be necessarily integrated into the browser engine implementer workflow. The core team overall goal should be bigger than just webplatform-tests/wpt. As such, in your list of potential changes requiring a RFC, I would add an another example something like "Adding a new repository for application-level or performance tests"

[plehegar]

btw, for the core team, what type of agreement did you have in mind?

[foolip]

I've put together https://github.com/foolip/rfcs as a strawman for how an RFC process might look.

btw, for the core team, what type of agreement did you have in mind?

That question was for @jgraham, but this is still TBD in my RFC repo. I think the core team would also be the group that somehow resolves RFC that do not reach a consensus.

[foolip]

@jgraham, I've incorporated https://github.com/web-platform-tests/wpt/issues/11473#issuecomment-413583544 into https://github.com/foolip/rfcs by referencing a core team which does not exist yet, and using some of your phrasing.

[foolip]

As for the core team, I agree with @plehegar in https://github.com/web-platform-tests/wpt/issues/11473#issuecomment-413597103 that the core team would manage web-platform-tests/*, not just web-platform-tests/wpt. I don't think it's important that we have rules about what number of members have which affiliation, and I think 5 is a reasonable group size.

The main challenge I see is how to populate the initial group. One idea is a vote among the top 100 wpt contributors, but I think that would suggest more prestige than we intend. But just coming to a silent agreement would also not be great.

So, the procedure I'd suggest is that we find a trusted volunteer who does not themself intend to be a member to do the following:

• Announce on public-test-infra that we're looking for up to 5 core team members, asking people to self-nominate or nominate others.
• If there are more than 5 candidates, the trusted volunteer and the candidates discuss to see if there is a natural way to trim the list. (For example, more than one candidate from the same company might not be necessary.) As a last resort, the trusted volunteer gets to make a suggestion based on any criteria or method (possibly voting) they like.
• The group of 5 is suggested on public-test-infra, with a 1 week period to object.
• If there are no objections, https://github.com/orgs/web-platform-tests/teams/core is created.

@web-platform-tests/admins and others, would you find that procedure acceptable?

[sideshowbarker]

@web-platform-tests/admins and others, would you find that procedure acceptable?

Sounds good to me

[marcoscaceres]

top 100 wpt contributors

You might want the top 100 in last 12 months maybe? From the full list, lots of people are inactive.

[foolip]

That might be better, yeah, although I hope no voting will be required. I'd leave that decision to the trusted volunteer.

[Ms2ger]

@web-platform-tests/admins and others, would you find that procedure acceptable?

SGTM, but maybe we can allow more than a week to object. We've gone without a core team for the best part of a decade, so I don't see a reason to rush it now.

[foolip]

Sure, we can give it a month, that would be fine.

[jgraham]

I think that procedure is acceptable, although I was imagining something less complex.

Appointing a trusted volunteer is challenging for theoretical and practical reasons. The theoretical reasons are just that the appointment of such a person is itself a fiat decision that needs to be made by someone or some group of people. Therefore it doesn't really reduce the number of meta decisions that need to be made, just causes a small amount of indirection. Practically finding a person who both has the relevant context and is neutral is hard. However if people feel the complexity is adding transparency then I'm happy to go ahead with that part of the plan.

[foolip]

Alright, I'll first try to find a volunteer, if I can't then obviously we're stuck at step 1. I agree that picking a volunteer is also a fiat decision, although it is one that's open to objection by anyone.

[gsnedders]

Doing it by voting alone seems likely to end up with @foolip, myself, and @jgraham among others on the core team; in a sense that's less than ideal because it means the majority of users of the project are essentially underrepresented (how many tests for the three of us write?). There's also the question of whether we want to encourage those currently less involved to become moreso.

[marcoscaceres]

Reading the responsibilities that @jgraham listed above, it makes sense that you, @foolip, and @jgraham would be on the core team. If you need a voice that represents test writers, there seem to be very good people in the top contributor list you could ping (e.g., Luna, Anne, Domenic, me, etc.)

[foolip]

@jgraham has sent out web-platform-tests Governance to public-test-infra, drafted by him, @marcoscaceres and me. The process is a little bit different (simpler) than what I described in https://github.com/web-platform-tests/wpt/issues/11473#issuecomment-425756781, so please read there and if you are interested fill out the self-nomination form.

[tobie]

Don't you want to extend this to the end of TPAC to make sure everyone sees it? Just a thought.

[foolip]

If someone shows up at TPAC that missed this discussion/announcement and would like to self-nominate, then sure, we'll include them in our effort to form a core team. But we'll want to do the "If possible a mutually agreeable decision is made from the pool of nominees" step during TPAC, so I don't think we should wait until the end to close the self-nomination form.

[annevk]

I got this clarified: there will be a 1 month objection period following any decision about who the core team should be with objections handled by @marcoscaceres: https://lists.w3.org/Archives/Public/public-test-infra/2018OctDec/0009.html. So whatever happens at TPAC is subject to scrutiny. This seems fairly reasonable to me.

[foolip]

@jgraham has announced the provisional membership of the initial core team and there is now a 1 month period in which objections to these members may be raised:
https://lists.w3.org/Archives/Public/public-test-infra/2018OctDec/0017.html

[tobie]

This team looks amazing. Congrats to all involved!

[gsnedders]

We now have a process and an RFC repo, so we can close this!