Browser-compat-data: Document whether WICG proposals are standards track

[ddbeck]

Proposals in the Web Incubator Community Group are of unclear value for the purposes of setting standards track data in BCD. We ought to ~make a decision about whether such proposals qualify as standards track and~ document how to interpret the standard track data for WICG proposals (either in the schema docs or data guidelines).

This has come up explicitly in a couple of recent PRs:

• https://github.com/mdn/browser-compat-data/pull/6030#discussion_r419700757
• https://github.com/mdn/browser-compat-data/pull/6089#issuecomment-624561842

And several PRs at least make mention of WICG.

There's a broader issue here that I'm avoiding addressing, which is that we don't really have a good definition of standards track in general. I've opened this issue because it's a narrow question that I think we can answer and document, but we have a stickier problem overall.

[ddbeck]

On #6030, I decided that a WICG proposal was not standards track, based on the information available to me. I might be wrong about this, but this is where I ended up one the question.

Firstly, I admit to having a very hazy idea of how standards come into existence and how W3C groups operate. Probably more than most web developers, but that's not saying much. And for WICG in particular, I'm not certain I had heard of it before the last couple of weeks. https://github.com/mdn/browser-compat-data/pull/6030#discussion_r419700757 was the first time I felt I needed to make a decision about one of its proposals for BCD. So I did some reading.

The WICG charter implies that WICG proposals have a sort of quasi-official status:

As proposals gain support and become more stable and mature they will be considered for migration to a W3C Working Group where they can be put on the Recommendation track with appropriate status and Intellectual Property (IP) considerations.

Dipping into the WICG admin repo turned up quite a few questions about the status of its specification artifacts from within the WICG:

• https://github.com/WICG/admin/issues/64
• https://github.com/WICG/admin/issues/94
• https://github.com/WICG/admin/issues/102
• https://github.com/w3c/tr-design/issues/186

I interpreted all this as meaning that WICG explainers and specifications are rough drafts that, if they're lucky, might some day ascend to standards track, via a working group. I could be wrong about the formality of it, but given the readily available evidence, I felt pretty good about choosing to regard WICG's work as non-standard.

[Elchi3]

I interpreted all this as meaning that WICG explainers and specifications are rough drafts that, if they're lucky, might some day ascend to standards track, via a working group.

This was my impression as well when looking at https://github.com/mdn/browser-compat-data/pull/6089#issuecomment-624561842. That PR was already merged, but I would have asked to change it to standards_track: false for now. I will ask around for further feedback, but I think we should document this policy for future reference, so that WICG drafts are not seen as standards. Thanks for filing this!

[bdekoz]

My preference is for incubator propoosals to be clearly marked as such. They are not standards, but proto-standards. Almost always, only one vendor supports these proposals......

[tantek]

From a strict interpretation, in the W3C at least, a specification must be at least a publicly published Working Draft (WD) by an active Working Group (WG) to be on an official "standards track", and thus that should be our condition for explicitly labeling a technology in a W3C document as "standards track".

At a minimum a specification must be accepted into a WG’s charter, and not just as a NOTE, in order to qualify to be standards track. However it’s not actually on that track (and citable as such) per se until the WG has agreed to publish it publicly as a WD.

By at least a WD, I’m explicitly saying yes it can also obviously be a Candidate Recommendation (CR), Proposed Recommendation (PR), or Recommedation (REC, or edited, or amended). If it’s an Obsolete Recommendation we should use the "Obsolete" label.

If it’s only in an Editor’s Draft or a WD (before a CR), that would be reasonable to label as "Experimental", as anything that’s not yet in a CR can "Expect behavior to change in the future."

If a document is for example only developed in a Community Group (CG) such as WICG, it is not standards track (CGs cannot make standards), and thus we should explicitly label any technologies there as "Non-standard", until such document makes its way into a WG and the WG publishes it as a WD, therefore publicly signaling that the WG has agreed to advance it onto the standards track.

For IETF and other orgs, I’ll let others chime in about what state a document must be in to transition between "non-standard" and "experimental" and "standards track", or "obsolete".

(Originally published at: https://tantek.com/2020/127/t2/)

[sideshowbarker]

Proposals in the Web Incubator Community Group are of unclear value for the purposes of setting standards track data in BCD.

I think we should be careful about how we use the term “proposal”. Any spec anywhere that hasn’t been implemented is just a proposal — even that spec is published by the W3C, or the IETF. So in that regard, there are many W3C Working Drafts and even Candidate Recommendations which are essentially just proposals.

Regardless of something is published, what makes it graduate into becoming a non-proposal is when it gets sufficient implementation support to be seen as usable by enough people who care.

So I would caution that — given what I know about the wide range of specs in development at the WICG — I think it’d be imprudent to make a single MDN/BCD policy for all WICG specs. I think that instead they just need to be considered case-by case.

Specifically I think what needs to considered most carefully for WICG specs — as with all specs published anywhere else — is implementation status: Given a particular WICG spec, has it been implemented in at least two browser engines? If not, are there public statements of support from browser-engine projects indicating they may be planning to implement? etc.

I interpreted all this as meaning that WICG explainers

I think the explainers are a very separate thing from the spec themselves, and it’d be best to just put those to the side when evaluating what policy to use for the actual WICG specs.

and specifications are rough drafts

Some WICG specs could be seen as rough drafts but many others are definitely much more than just rough drafts.

that, if they're lucky, might some day ascend to standards track

The thing is, the WICG can be considered part of the W3C standards track. It was certainly created and designed to be part of standards track.

The WICG has a long and complicated backstory, but one aspect that played into it was opposition among some W3C member organizations and staff to having work on so-called “incubations” taking place in W3C working groups. So the WICG was created as a place for work that fell into that class of ”incubation” work which we didn’t have consensus should be taking place in working groups any longer. And in the minds of some people at least, they hoped that’d provide a clear, clean separation from “incubation” stuff and other stuff that was further along, more mature.

But the reality is that things are not the clear and we don’t actually have a clean separation in practice. What I mean is, we still have some work taking place, in some W3C working groups, on specs that are just as much “incubations” as anything in the WICG. And we have some WICG specs that are much more mature and further along on a trajectory to becoming standards than some specs in some W3C working groups are.

via a working group. I could be wrong about the formality of it, but given the readily available evidence, I felt pretty good about choosing to regard WICG's work as non-standard.

While it is certainly accurate to say that all WICG specs are strictly non-standard in the typical taxonomy used by formal standards-development organizations to define what a standard is, that’s something quite different from standards track.

Clearly, any given spec can be seen as being “on the standards track” without necessarily being an actual standard yet.

So, what I am asserting is that a number of WICG specs are in fact just as much “on the standards track” as some specs in W3C working groups are.

I think we should document this policy for future reference, so that WICG drafts are not seen as standards.

I concede that WICG specs can’t be considered standards. But for the reasons outlined above, I assert the same can be said about many specs in W3C working groups. It seems like what we need to do instead is to distinguish between stuff that can be seen as “standards track” and stuff that can’t. And by the some criteria that the standards community uses in practice to make that determination, some WICG specs can be considered “standards track".

[sideshowbarker]

If a document is for example only developed in a Community Group (CG) such as WICG, it is not standards track (CGs cannot make standards)

Even if one accepts the “CGs cannot make standards” premise, from that it doesn’t necessarily follow that a spec in a CG can’t be “standards track”. The WICG was specifically created to have a place in the W3C standards track — while, admittedly, a place early along in the standards track.

and thus we should explicitly label any technologies there as "Non-standard"

I think labeling WICG specs “non-standard” doesn’t resolve the problem we’re trying to resolve — which is more precisely about whether or not any of them can be labeled “standards track” in the specific https://github.com/mdn/browser-compat-data/blob/master/schemas/compat-data-schema.md#status-information taxonomy we’re using for BCD.

until such document makes its way into a WG and the WG publishes it as a WD, therefore publicly signaling that the WG has agreed to advance it onto the standards track

I think it would be a mistake for us to restrict the “standards track” designation to being allowed only for WDs in W3C working groups but never for WICG work.

For the sake of comparison, consider the case that for many years, many people in the standards community considered all WHATWG specs to be non-standards and non-standards-track. Some people still consider WHATWG specs to be non-standards-track.

So it seems like for identifying whether something is “standards track”, we need to have some criteria which is less restrictive than “makes its way into a WG and the WG publishes it as a WD”.

Also, consider the case of WebAssembly, a CG product that was already implemented in all major browser engines before any W3C working group even existed for it. A WG was created for it after the fact. And the scope of the WG related to it is essentially explicitly not technical. The technical work for WebAssembly still takes place in the CG. What the WG adds is patent commitments and wider recognition for it within the industry.

So we don’t consider the WebAssembly spec to be standard simply because a W3C WG has (re)published the spec for it. Or certainly I would think we shouldn’t consider the existence of a WG for it to have been necessary for it to be considered a standard. It would be a standard in practice even if a W3C working group never existed for it.

[sideshowbarker]

Maybe we should consider just completely dropping the standard_track key.

I think we’ll never arrive at a clear, simple single agreed-upon definition of what “standards track” means. It’s always going to be subjective; we can’t objectively, deterministically identify whether something is “standards track”.

And we should also (re)consider what purpose is served by having standard_track key in BCD to begin with. Is it being used beyond just for the generated indicators in the MDN compat tables?

For the purpose of giving such indicators, maybe what’d work better is to just show a label among W3C, WHATWG, WICG, ECMA (or TC39), IETF, and Khronos. Or the corresponding logos.

That would be an objective indicator. For MDN it would be one signal, among others, that we give to readers in order for them to make their own assessments. An additional data point.

The labels/logos could be generated, deterministically, just based on the spec URLs. Readers of course could get the same information by looking at the URLs themselves. But showing the labels/logos would save readers the time/trouble they’d need to spend hunting for the URL.

Also, there are some cases (specifically, CSSWG, FXTF and Houdini drafts) that are W3C products but not published in w3.org space. So readers looking just at the URL for one of those cases may not necessarily have the prior knowledge needed to recognize them as W3C products.

Anyway, for the sake of comparison, note how Can I Use handles labeling. Specifically, Can I Use doesn’t have any equivalent of a “standards track” designation. Instead it has a “status” indicator which is one of LS, REC, PR, CR, WD, Other, and Unofficial —
https://github.com/Fyrd/caniuse/blob/master/CONTRIBUTING.md#supported-changes

And when you go to a Can I Use page such as https://caniuse.com/#feat=fontface, that status indicator is shown right next to the feature title.

I’m definitely not suggesting we adopt a LS/REC/PR/CR/WD/Other/Unofficial status indicator like that. I think such spec-status information isn’t very useful to readers — and in some cases can even be very misleading. And “Unofficial” is subjective.

What I am suggesting instead is that at MDN (and actually, at Can I Use too), what could be more helpful to readers is a W3C, WHATWG, WICG, ECMA/TC39, IETF, or Khronos label/logo — shown in a place similar to where Can I Use shows their status indicator now.

[Elchi3]

Some more thoughts after having slept on this.

Thanks for your feedback @tantek and @sideshowbarker :+1: Now I understand a lot more about the different document statuses and also how different organizations, browser vendors, and/or standards bodies might have different opinions about this. The "strict interpretation, in the W3C at least" is interesting and one to keep in mind, but we probably need to see how this works across the different standards.

That said, for MDN and BCD, we need to see what is most useful to signal to the readers (which are mostly web developers) on the MDN docs site, caniuse, and in tools like VS Code. So, it probably makes sense to think this though from that perspective and not from a single standards organization.

Maybe we should consider just completely dropping the standard_track key.

Yes, I think this is what it comes down to, because it doesn't provide enough nuance. And I actually proposed this some time ago after discussions in the MDN PAB. See the outcome in https://github.com/mdn/browser-compat-data/issues/1531.

And we should also (re)consider what purpose is served by having standard_track key in BCD to begin with. Is it being used beyond just for the generated indicators in the MDN compat tables?

For the purpose of giving such indicators, maybe what’d work better is to just show a label among W3C, WHATWG, WICG, ECMA (or TC39), IETF, and Khronos. Or the corresponding logos.

This is a good point. Rather than having a boolean flag (standard track: yes or no) we could indicate where the spec is from. If all features in BCD have spec_urls (like it is the plan), the we could display such logos (or at least any indication) depending on the spec_url domain. We can then add general guidance to what it means if a spec is from a WICG document vs a W3C recommendation and sometimes maybe also specific guidance if a spec is controversial, or mark as experimental. For that, @tantek's guidelines are a start.

For the feature in question, chromestatus actually has "Consensus and standardization" (https://www.chromestatus.com/features/5666250908762112) which I think is in fact another useful aspect to MDN readers, too, so that they can properly judge the status of the specification. On MDN at least, we might want to look into how to do something similar. For example, linking to positioning on standards. However, we probably don't need this machine-readable in BCD, I think spec_url is enough long term.

I’m definitely not suggesting we adopt a LS/REC/PR/CR/WD/Other/Unofficial status indicator like that. I think such spec-status information isn’t very useful to readers — and in some cases can even be very misleading. And “Unofficial” is subjective.

I agree to this as well. Again, "Consensus and standardization" (cf. chromestatus) seems much more useful information than these statuses.

[cwilso]

I'd like to suggest that you not consider "WICG" as a separate category (or the name of a category). Anything true of a WICG spec's status is likely true from another CG as well - it's an incubation, but it's not an accepted consensus-driven work. I'd expect specs in the Immersive Web Community Group would be marked as incubation - I'd expect specs from the new Privacy CG to have the same status.

As pointed out, "this is not standards track" is a bit strongly worded - I agree with what @sideshowbarker said about "incubations are still kind of possibly early standards track" - but perhaps the right way to represent this is to have a "this is an incubation" label instead. That label should be applied to anything that's in a CG at the W3C - and probably anything that's lingering in a PR in the WHATWG as well. It probably does NOT apply to most WG specs, even if they're not CR or Rec status yet, unless the WG considers them an incubation.

[jpmedley]

The whole source of this debate is that we're trying to apply a binary element to a situation that isn't binary. Let's pretend the current data set doesn't cover this at all. What would we do? We'd look at the thing we're trying to capture and design according to that. Instead of trying to cram a square pet in a round whole, we'd just design square hole.

What might that square hole look like? I'm thinking it's an enum instead of a boolean. Maybe it's called standards_disposition (I'm not in love with the name.) It would have values like "standards_track", "incubation", and "standard".

This is somewhat simpler than the reality. I would:

• Opt for clarity to web developers who may or may not understand anything about standards processes and groups.
• Provide a path down the rabbit hole for people who do want to know more.

This would separate what developers need to know to move forward from all the messiness behind. Yet they could still find that information is it were relevant to them or they were just curious.

For example, I wouldn't ask developers to understand what WHATWG and WICG. Communicating that their products are non-standard is enough. By the way, if we opted for simply naming the group, we'd still need to clarify which ECMA or IETF items are non-standard.

[cwilso]

I think it would be fine to do something like what you're suggesting - denote "incubation" vs "standards-track draft" (aka under control of a WG if at the W3C) or "standard" (Aka REC or in core WHATWG specs).

(Note that you shouldn't equate WHATWG and WICG above - WICG only has incubations, as does any Community Group at the W3C, but WHATWG has official standards (e.g. DOM and HTML) as well as incubations (in PRs, usually).

[ddbeck]

Thanks to everyone on this issue for your feedback.

It sounds like there's a lot more nuance than I first guessed and I drew overbroad conclusions from a small sample.

Longer term, I'd like to resolve the broader issue of how we represent standardization status in BCD. In the nearer term, I've lightly modified the original issue description to strike out the "make a decision" aspect of this. Instead, I'd like to document guidance for BCD contributors to help them understand some of this nuance.

[foolip]

This came up in https://github.com/mdn/browser-compat-data/pull/7436#pullrequestreview-537698053. https://wicg.github.io/element-timing/ is a case which I think doesn't warrant setting standard_track to true.

I wouldn't say the same for https://wicg.github.io/speech-api/ though even though its status is the same, https://developer.mozilla.org/en-US/docs/Web/API/SpeechSynthesis#Browser_compatibility is very green and there's no reason to avoid it based on where the spec is hosted. It was just moved to the WICG because there was no other existing CG or WG which made sense for it.