Browser-compat-data: Choose and document a policy for flagged features

[ddbeck]

It's been suggested a couple of times now that we should discard data about features behind preferences or flags after those features have been turned on by default. We ought to make a decision about this and document the decision.

For example, see @jpmedley's https://github.com/mdn/browser-compat-data/issues/2992#issuecomment-430999779 on Chrome preferences or @chrisdavidmills's PR #3314, where he tried to delete old data involving Firefox preferences.

[ddbeck]

My take: I think we should consider the distinction between what data this project provides to consumers (i.e., what data we accept and keep in this repo) and what those consumers choose to present to their users (e.g., what data is shown in the tables on MDN pages). For example, if preference information is confusing to MDN readers, then that's an argument for not showing that data to MDN readers, not necessarily an argument for removing that data from the repo.

From a practical perspective, I think instituting a policy of dropping data when features become default will complicate reviewing PRs. As it stands, data removals generally indicate that data was wrong or misplaced or incomplete, not too old for the Chrome team to care about or whatever.

My vote is that we should include any data that's true and then make smart decisions about how that data is used. If there are certain patterns of data usage that are especially desirable or common (for instance, a subset of the data that excludes flagged features), then we might consider providing an API that only exposes that abbreviated data set, instead of trying to shrink the data itself.

[Elchi3]

I completely agree that there is a differentiation between providing data and displaying data and that we shouldn't necessarily model all data in the way we want it to be displayed on MDN docs.
However, if the data isn't interesting to any consumer, we need to see if it is worth the effort of keeping and maintaining such data. We probably also ought to ask who is our audience here. I have heard anecdotally that flags and such aren't really useful to web developers and they aren't really interested in them. I'm not sure if this is true. It is possibly more of a power user, browser engineer or spec author information, though. We could user test this, if we think it is important to find out the audience for this I guess. We could then also ask these groups if they are interested in historic information at all.

But OK, I'm trying to ignore the displaying and usefulness aspect and will try to just focus on the data and its quality and the maintenance of it.

Maintainability:

From a practical perspective, I think instituting a policy of dropping data when features become default will complicate reviewing PRs.

I'm not yet convinced this is always the case. I think there is a higher effort right now when we keep the "dead flags": It is more complicated to come up with good version ranges than it would be when just removing the flag info and adding a simple "version_added". The diff right now is about going from one version entry to at least two, meaning that the existing data needs to be touched and array data is added ("version ranges"). The diff when removing dead flags is more clear imo.

Data quality:

For the version ranges we still have the sorting issue unresolved which makes the data inconsistent and therefore machines can't really deal with it very well (https://github.com/mdn/browser-compat-data/issues/1596). It should be reverse-chronological, but no linter has enforced this and there are cases where we don't want it to be strictly reverse-chronological iirc.

Also, there is probably no automation (like confluence) or anything that can help us verifying that info for dead flags is correct. So, any historic dead flag info is curated by humans now and going forward.

[chrisdavidmills]

I personally think dead flag information is pointless after the related features have been enabled by default.

I'd prefer to get rid of the information after said feature has been enabled. It is useful while the feature is still behind a flag and a small number of early adopters want to test it, but after it is enabled, I'm pretty sure no-one cares.

I could live with keeping the data in BCD, but no longer displaying it by default after the feature is enabled by default.

When we move further along with the idea of BCD display personalization for logged in users, we could maybe provide a checkbox to display this data again, in case anyone really wants to see it.

[estelle]

Once a feature is enabled by default, the flag information is of no use, with the exception of historical data for the 5 people world wide who care which browser implemented a feature first (rather than releasing a feature first). This is not enough of a use case to keep the data in the files, and definitely not enough of a use case to surface that informationto our users. If someone is really interested in that information, searching bugzilla will provide them with that information as would reviewing all the commits on the feature itself in the github repo.

TL;DR; Surfacing dead flag info to our users is detriment to user experience.I would prefer if we didn't even keep that information in the BCD files. As stated above, it's not necessary.

[a2sheppy]

It's pretty well known that I don't like getting rid of data. I think the value of historical information is generally underappreciated, etc. However, in this case, I think it does make sense that a time does come where it makes sense to get rid of obsolete information about things like prefs that enable a feature.

I think we should use rules something like this:

• When the flag is removed from the browser (not just enabled by default, but actually removed entirely), remove all mention of the flag from BCD.
• When the flag is switched to enabled by default, clean up the flags section to remove the dependency on the flag, but we should have a way to include the fact that the flag exists and can be used to turn the feature off.

[jpmedley]

I've tried to use personal persuasion on this issue, there are still people unconvinced. With some very rare exceptions, I don't put flags in the data I submit. This is not my personal opinion. I have been explicitly instructed not to publicize flag information.

When a flag is not explicitly is not really a useful threshold. For Chrome, flag removals are the kind of thing that happen on Friday afternoon when it's too late to start working on the new Wonder Widget API. I've had conversations about flags where the engineer basically said, 'woops! I forgot that was still around.'

I don't really care about dead flags where other browsers are concerned. Where Chrome is concerned, this is useless information. It is noise.

[a2sheppy]

The problem of course is if you add data for an API that is disabled by
default in a given release; then it only exists in that version if the flag
is set. The compat info is actually wrong in that case, from the web
developer's point of view.

--
Eric Shepherd
Senior Technical Writer
Mozilla Developer Network
https://developer.mozilla.org/
Blog: http://www bitstampede.com/
Twitter: https://twitter.com/sheppy

On January 22, 2019 at 10:29:20 AM, Joe Medley ([email protected])
wrote:

I've tried to use personal persuasion on this issue, there are still
people unconvinced. With some very rare exceptions, I don't put flags in
the data I submit. This is not my personal opinion. I have been explicitly
instructed not to publicize flag information.

When a flag is not explicitly is not really a useful threshold. For
Chrome, flag removals are the kind of thing that happen on Friday afternoon
when it's too late to start working on the new Wonder Widget API. I've had
conversations about flags where the engineer basically said, 'woops! I
forgot that was still around.'

I don't really care about dead flags where other browsers are concerned.
Where Chrome is concerned, this is useless information. It is noise.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/mdn/browser-compat-data/issues/3318#issuecomment-456441565,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABkL3_YAui1LUDGA__ZrAcNrmXms7ETfks5vFy5QgaJpZM4aLLvk
.

[jpmedley]

That is exactly the point. By policy, Google does NOT use MDN as a
distribution channel for that type of information.
Joe Medley | Technical Writer, Chrome DevRel | [email protected] |
816-678-7195
If an API's not documented it doesn't exist.

On Tue, Jan 22, 2019 at 11:58 AM Eric Shepherd [e-mail]
wrote:

The problem of course is if you add data for an API that is disabled by
default in a given release; then it only exists in that version if the flag
is set. The compat info is actually wrong in that case, from the web
developer's point of view.

--
Eric Shepherd
Senior Technical Writer
Mozilla Developer Network
https://developer.mozilla.org/
Blog: http://www bitstampede.com/
Twitter: https://twitter.com/sheppy

On January 22, 2019 at 10:29:20 AM, Joe Medley ([email protected])
wrote:

I've tried to use personal persuasion on this issue, there are still
people unconvinced. With some very rare exceptions, I don't put flags in
the data I submit. This is not my personal opinion. I have been
explicitly
instructed not to publicize flag information.

When a flag is not explicitly is not really a useful threshold. For
Chrome, flag removals are the kind of thing that happen on Friday
afternoon
when it's too late to start working on the new Wonder Widget API. I've
had
conversations about flags where the engineer basically said, 'woops! I
forgot that was still around.'

I don't really care about dead flags where other browsers are concerned.
Where Chrome is concerned, this is useless information. It is noise.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<
https://github.com/mdn/browser-compat-data/issues/3318#issuecomment-456441565
,
or mute the thread
<
https://github.com/notifications/unsubscribe-auth/ABkL3_YAui1LUDGA__ZrAcNrmXms7ETfks5vFy5QgaJpZM4aLLvk

.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/mdn/browser-compat-data/issues/3318#issuecomment-456476790,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AH0viwaJ59_gCcmA2_FyX7xDV3YlMEDCks5vF0NSgaJpZM4aLLvk
.

[a2sheppy]

Obviously, that's up to Google to decide. In that case, a feature should
not be considered to be present in Chrome until the flag is on by default.
That's something that could be aided by my proposal to add explicit support
for channels (beta, nightly, etc) to BCD. So that if it's enabled by
default in Canary, it could be listed as compatible in Canary 75 and later
but only Canary.

--
Eric Shepherd
Senior Technical Writer
Mozilla Developer Network
https://developer.mozilla.org/
Blog: http://www bitstampede.com/
Twitter: https://twitter.com/sheppy

On January 22, 2019 at 1:25:16 PM, Joe Medley ([email protected])
wrote:

That is exactly the point. By policy, Google does NOT use MDN as a
distribution channel for that type of information.
Joe Medley | Technical Writer, Chrome DevRel | [email protected] |
816-678-7195
If an API's not documented it doesn't exist.

On Tue, Jan 22, 2019 at 11:58 AM Eric Shepherd [e-mail]
wrote:

The problem of course is if you add data for an API that is disabled by
default in a given release; then it only exists in that version if the
flag
is set. The compat info is actually wrong in that case, from the web
developer's point of view.

--
Eric Shepherd
Senior Technical Writer
Mozilla Developer Network
https://developer.mozilla.org/
Blog: http://www bitstampede.com/
Twitter: https://twitter.com/sheppy

On January 22, 2019 at 10:29:20 AM, Joe Medley ([email protected]
)
wrote:

I've tried to use personal persuasion on this issue, there are still
people unconvinced. With some very rare exceptions, I don't put flags
in
the data I submit. This is not my personal opinion. I have been
explicitly
instructed not to publicize flag information.

When a flag is not explicitly is not really a useful threshold. For
Chrome, flag removals are the kind of thing that happen on Friday
afternoon
when it's too late to start working on the new Wonder Widget API. I've
had
conversations about flags where the engineer basically said, 'woops! I
forgot that was still around.'

I don't really care about dead flags where other browsers are
concerned.
Where Chrome is concerned, this is useless information. It is noise.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<

https://github.com/mdn/browser-compat-data/issues/3318#issuecomment-456441565

,
or mute the thread
<

https://github.com/notifications/unsubscribe-auth/ABkL3_YAui1LUDGA__ZrAcNrmXms7ETfks5vFy5QgaJpZM4aLLvk
>

.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<
https://github.com/mdn/browser-compat-data/issues/3318#issuecomment-456476790
,
or mute the thread
<
https://github.com/notifications/unsubscribe-auth/AH0viwaJ59_gCcmA2_FyX7xDV3YlMEDCks5vF0NSgaJpZM4aLLvk

.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/mdn/browser-compat-data/issues/3318#issuecomment-456508445,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABkL3zPd_UvJ5077ayIwjrzzta8vyvPJks5vF1eMgaJpZM4aLLvk
.

[ddbeck]

I really appreciate everyone's commentary on this. It seems like the overall inclination is toward dropping at least some of the flag data. This bums me out but if we're headed that way, I think it leads to some more questions:

Can we confirm that the historic flag data is, in fact, useless? Do we know anything concrete about BCD consumers and users and flag data? If not, can we come up with a plan to find out? I'd rather not foreclose on use cases for BCD on intuition alone.

If we choose to keep current but not historic flag data, are we concerned about creating new sources of confusion? And can we come up with a coherent plan for merging flag data deletions?

Imagine this hypothetical: today, some feature in some browser version 47 (the current release) is behind a preference and that's what we've got in our data. A few weeks from now, that feature is turned on by default in 50 (a canary/nightly release) and we get a PR that replaces the data applicable to 47-48-49 with the data for 50.

Do we merge that data immediately? If we merge it immediately, are we OK with our data implying that there's less support for that feature a few weeks from now than we're reporting today, even though the opposite is true? (In more concrete terms, this might look like a trick on MDN readers: just as you approach point of transition from not-default to default—when a developer might be especially interested in trying out a feature—we yank that data.)

If we don't accept the PR immediately, then what? When do we accept and what's the criteria? After some period of stability? What do we do with the PR meanwhile? Close the PR and ask for resubmission later or leave it open? Build some kind of automation to delete flag data when the time is right?

[a2sheppy]

@ddbeck -- While I personally don't usually like to remove data (I'm a big proponent of keeping the history intact, always), I'm willing to accept that this might be an area where we look at removing some stuff. My preference is not to remove data from BCD, but to leave it out during presentation. But if the decision is made to remove it, I want to moderate how much we remove, and be sure we only remove it when the impact is minimized as much as possible, hence my suggestions here.

I feel that the flag information needs to be kept at a minimum until the change that removes the flag has gone all the way to "shipped," or ideally a couple of releases past that point. I'd prefer a year or two after release, but I doubt I'd get many takers for that idea.

I feel that if we're going to start removing stuff at some point after we feel it's no longer relevant, we will have to come up with a system for tracking these things to be sure they happen at the appointed times.

[jpmedley]

"Can we confirm that the historic flag data is, in fact, useless? Do we
know anything concrete about BCD consumers and users and flag data? If not,
can we come up with a plan to find out? I'd rather not foreclose on use
cases for BCD on intuition alone."

I don't know how many ways I can say this. Once a feature goes to
production, the flag information is useless. Google doesn't tell people to
use it. If a developer asks us for help with one of these features our
answer is upgrade.

"If we choose to keep current but not historic flag data, are we concerned
about creating new sources of confusion? And can we come up with a coherent
plan for merging flag data deletions?

"Imagine this hypothetical: today, some feature in some browser version 47
(the current release) is behind a preference and that's what we've got in
our data. A few weeks from now, that feature is turned on by default in 50
(a canary/nightly release) and we get a PR that replaces the data
applicable to 47-48-49 with the data for 50."

As a standard practice, I would not submit such data before Chrome 50
shipped to stable. Submitting when it goes to beta is generally pretty safe
(for us), but I generally don't do it. I absolutely don't do it when a
feature is in Canary. This has a very simple reason: features in Canary and
beta are not guaranteed to be in stable. Not only do I not submit this
information, I actively discourage others from doing so.

"I feel that the flag information needs to be kept at a minimum until the
change that removes the flag has gone all the way to "shipped," or ideally
a couple of releases past that point. I'd prefer a year or two after
release, but I doubt I'd get many takers for that idea."

As I said above, when a flag is explicitly removed is not really a useful
threshold for Chrome. Flag removals are sporadic, and sometimes only happed
when someone points out that it's still around.

Joe Medley | Technical Writer, Chrome DevRel | [email protected] |
816-678-7195
If an API's not documented it doesn't exist.

On Wed, Jan 23, 2019 at 8:12 AM Daniel D. Beck [e-mail]
wrote:

I really appreciate everyone's commentary on this. It seems like the
overall inclination is toward dropping at least some of the flag data. This
bums me out but if we're headed that way, I think it leads to some more
questions:

Can we confirm that the historic flag data is, in fact, useless? Do we
know anything concrete about BCD consumers and users and flag data? If not,
can we come up with a plan to find out? I'd rather not foreclose on use
cases for BCD on intuition alone.

If we choose to keep current but not historic flag data, are we concerned
about creating new sources of confusion? And can we come up with a coherent
plan for merging flag data deletions?

Imagine this hypothetical: today, some feature in some browser version 47
(the current release) is behind a preference and that's what we've got in
our data. A few weeks from now, that feature is turned on by default in 50
(a canary/nightly release) and we get a PR that replaces the data
applicable to 47-48-49 with the data for 50.

Do we merge that data immediately? If we merge it immediately, are we OK
with our data implying that there's less support for that feature a few
weeks from now than we're reporting today, even though the opposite is
true? (In more concrete terms, this might look like a trick on MDN readers:
just as you approach point of transition from not-default to default—when a
developer might be especially interested in trying out a feature—we yank
that data.)

If we don't accept the PR immediately, then what? When do we accept and
what's the criteria? After some period of stability? What do we do with the
PR meanwhile? Close the PR and ask for resubmission later or leave it open?
Build some kind of automation to delete flag data when the time is right?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/mdn/browser-compat-data/issues/3318#issuecomment-456796079,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AH0vixwTmzfAZyTKAh8AWm3ijWbTJqSlks5vGF-sgaJpZM4aLLvk
.

[ddbeck]

@jpmedley

I don't know how many ways I can say this. Once a feature goes to production, the flag information is useless. Google doesn't tell people to use it. If a developer asks us for help with one of these features our answer is upgrade.

A few things about your reply:

1. There _are_ other browsers than Chrome. I specifically didn't name a browser in my hypothetical. I wrote the initial issue in direct response to a case involving a Firefox preference. I think we're going to have to deal with the question of what flags mean for more than just Chrome.
2. Though the data may not be actionable for web developers, I'm not convinced that Chrome's position nullfies all uses for the data, particularly the history itself (yes, I could go research when some features were first experimental in Chrome or Firefox, but it'd be a heck of a lot easier to query BCD for that). I believe that that kind of use is unlikely, but I'd like us to at least look around a little before we declaring it so.

3. Does this kind of structure not characterize Chrome's position on flags?

   {
       "chrome": [
           {
               "version_added": "X"
           },
           {
               "version_removed": "X",
               "version_added": "X – 4",
               "flags": [
                   ...
               ]
           }
       ]
   }
   

[a2sheppy]

If flag information is less useful for Chrome, or is only useful in limited ways, that's fine. In Firefox, it's generally much more useful. We also like to consider the fact that developers may need to test their code on older browsers for compatibility reasons, and that may involve checking and adjusting configuration options both to enable needed features and to ensure that they fail gracefully when those features aren't available.

[Elchi3]

Here are the features that likely have "dead flags" https://gist.github.com/Elchi3/36e7e823027a91561e9d5891955f096a

[Elchi3]

The same query but with the actual support data in question that we want to remove if would do this https://gist.github.com/Elchi3/50468ab812d4c83539404526ad17066e

[estelle]

I would like to explain my opposition to including flag data indefinitely in the form of an example.

Please look at https://developer.mozilla.org/en-US/docs/Web/CSS/animation-delay

Getting rid of the following notes will negatively impact no one, but will reduce the confusion created by all the notes:

44 - Prefixed Disabled
Prefixed Implemented with the vendor prefix: -webkit-
Disabled From version 44: this feature is behind the layout.css.prefixes.webkit preference (needs to be set to true). To change preferences in Firefox, visit about:config.

As currently displayed, with 16*, I think it looks to the average user that FF started supporting animation-delay in FF16, even though support started with FF5, prefixed. But that's not the only issue with it.

This historical data, as presented or not presented,in this case, is more harmful than helpful.

If the dev clicks on the "16*", they see a bunch of information, including the -webkit- flag from FF44 to 48. Who does this help? Who needs to know that FF supported the property -webkit-animation-delay in FF 44 to 48, but only if the user set the flag to true. No web developer is going to rely on their users switching the flag, no users are using FF 44 to 48, and no devs are testing 44 to 48 anyhow.

Since FF 49, -webkit-animation-delay has been supported. The fact that -webkit- prefixed version is supported in Moz is helpful to people who are concerned that their legacy site prefixed all the animation properties. They may also care if they used (-webkit-animation-delay: 2s) as part of a @supports.

In other words, yes, we should keep this latter information, but the legacy flag information is NOT necessary.

There are four notes under this property and an asterisk. It is overwhelming. A quick glance at the BCD, without opening the note, makes it look like FF still has an issue with this property. I think we need to START by removing dead flag information - say anything before 63. But there is still a lot of other work that needs to be addressed to make our BCDs more intuitive and therefore more competitive with resources like CanIUse. If the cell read:
5-x, 16 [*], 57

It might make it look busier, but it would likely be more intuitive.

[a2sheppy]

But, once again, it’s important to note there’s a difference between
retaining the data and presenting the data. I want us to keep the data
available, but come up with ways to adjust how much of it is displayed on a
standard MDN page to keep the clutter to a minimum while maximizing
usefulness.

On January 26, 2019 at 12:30:47 AM, Estelle Weyl ([email protected])
wrote:

I would like to explain my opposition to including flag data indefinitely
in the form of an example.

Eric Shepherd
Senior Technical Writer
MDN Web Docs https://developer.mozilla.org/
Blog: https://www.bitstampede.com/

[chrisdavidmills]

I still think that for the short term it would make sense to keep all the data, but by default not display it after it is enabled by default.

In the longer term we would include an option somewhere to display the full historic pref data if desired, as part of the personalization work we are talking about.

We also ought to do some research to see if any of our users actually find this data useful, and what they use it for.

[ExE-Boss]

I believe that we should adjust the {{Compat}} macro to hide compatibility data which are behind feature flags for old Firefox versions (anything older than current ESR) by default.

[jpmedley]

If this data were only on MDN, I wouldn't object. If it shouldn't be on
MDN, it shouldn't be in visual studio. The only way we have to control that
is to keep it out of BCD completely.
Joe Medley | Technical Writer, Chrome DevRel | [email protected] |
816-678-7195
If an API's not documented it doesn't exist.

On Mon, Jan 28, 2019 at 12:36 AM Chris Mills [e-mail]
wrote:

I still think that for the short term it would make sense to keep all the
data, but by default not display it after it is enabled by default.

In the longer term we would include an option somewhere to display the
full historic pref data if desired, as part of the personalization work we
are talking about.

We also ought to do some research to see if any of our users actually find
this data useful, and what they use it for.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/mdn/browser-compat-data/issues/3318#issuecomment-458039108,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AH0vi78bhLaIybpmGNHP8DetC2BJ-jJeks5vHraHgaJpZM4aLLvk
.

[a2sheppy]

But it’s trivial for anyone to choose what they feel is useful to display.
We can include the information and Visual Studio can leave it out. I don’t
see a problem here. If Google has information they do not feel should be
disseminated, we can certainly omit it from the data, but that doesn’t
require establishing a global policy that has to be followed for every
browser.

On February 1, 2019 at 12:52:46 PM, Joe Medley ([email protected])
wrote:

If this data were only on MDN, I wouldn't object. If it shouldn't be on
MDN, it shouldn't be in visual studio. The only way we have to control that
is to keep it out of BCD completely.
Joe Medley | Technical Writer, Chrome DevRel | [email protected] |
816-678-7195
If an API's not documented it doesn't exist.

On Mon, Jan 28, 2019 at 12:36 AM Chris Mills [e-mail]
wrote:

I still think that for the short term it would make sense to keep all the
data, but by default not display it after it is enabled by default.

In the longer term we would include an option somewhere to display the
full historic pref data if desired, as part of the personalization work we
are talking about.

We also ought to do some research to see if any of our users actually find
this data useful, and what they use it for.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<
https://github.com/mdn/browser-compat-data/issues/3318#issuecomment-458039108
,
or mute the thread
<
https://github.com/notifications/unsubscribe-auth/AH0vi78bhLaIybpmGNHP8DetC2BJ-jJeks5vHraHgaJpZM4aLLvk

.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/mdn/browser-compat-data/issues/3318#issuecomment-459808632,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABkL33WbIWUCP1PT2wbG9vWz6LwkwSBTks5vJH7tgaJpZM4aLLvk
.

Eric Shepherd
Senior Technical Writer
MDN Web Docs https://developer.mozilla.org/
Blog: https://www.bitstampede.com/

[ddbeck]

Since I think everyone has had a chance to discuss this (and I've started to see a bit of rerunning discussion here), I'd like to start wrapping things up by articulating an actual policy and move toward documenting it. I think there's an approach that might satisfy some of the competing interests expressed in this issue. Here's what I propose:

Only record version numbers for flags where both a) the flag exists and b) the flag is off by default.

For example, suppose some browser has a history like this:

• Some Feature is unsupported from versions 1-5.
• Some Feature flag is added in version 6. It is off by default.
• Some Feature is enabled by default in version 8.
• Some Feature flag is removed in version 10.

In this case, we'd record:

  "some_browser": [
    {
      "version_added": "8",
    },
    {
      "version_removed": "8",
      "version_added": "6",
      "flags": [...]
    }
  ]

Benefits to this approach:

• We only concern ourselves with the maximal extent of feature support. In other words, we're focusing only on the case of a browser's most-widely available support for a feature, not every way a feature may or may not be supported in a release.
• We don't have any overlapping version ranges for on-by-default status (i.e., only one entry would concern the current version of a browser's support).
• We don't have to manage any kind of flag removal process (for example, we don't have to worry about keeping Firefox ESR flag data around longer than other releases).
• We preserve partial history about flags (e.g., we see when a feature was in development, though not necessarily when that feature ceased to be optional).
• This would appear to satisfy the Chromium's team desire to disavow flags once a feature has been switched on by default. Alternatively, we can carve out an exception for Chromium to cease documenting their flags entirely, since I think we could enforce this in the schema.

Disadvantages to this approach:

• Contributors and maintainers will have to retrain themselves to add version_removed entries for on-by-default features.
• Very old flag data may still exist and conusmers of the data can continue to choose to present it to their users.
• Some existing data doesn't correspond to this approach and will need updating (either as a big push or incrementally).

Alternatives I considered proposing and rejected:

• The status quo of having data for concurrent live features and dead flags (e.g., open ended version_added ranges for flags which are now on by default). Rejected because it seems untenable given the strong desire to do _something_. This seems like the best backup policy, though.
• Deleting flag data after a feature is on-by-default. Rejected because it would require managing the deletion process independently of adding new data (e.g., waiting for an ESR release before deleting Firefox flag data or waiting for a feature on any browser to proceed through beta to stable). It doesn't sound like anyone is actually volunteering to do this work.
• Deleting _all_ flag data and not including it in BCD at all. Rejected because it appears there's an interest in some flag data, if not all (e.g., before a feature is available by default in any version).

I'd be interested in seeing some 👍 / 👎 reactions on this proposal, to see if it's anywhere near resolving the issue.

[chrisdavidmills]

I think this sounds like a well thought out and sensible way forward on this. It is nice to have a set of guidelines on this. Browser vendors can choose to provide their flag information or not.

This also works in situations where you have multiple flag configs. You can just list multiple support data, with version added/removed indicating where the flag situation changed.

Related question: In terms of documenting bugs where the flag is already flipped to true by default (e.g. feature is enabled by default), but then the flag itself is removed (i.e. you can no longer use it to disable a feature if desired), we already don't document such instances in BCD to my knowledge (although correct me if I'm wrong), and I think we also shouldn't document these anywhere on MDN anymore (I have been adding them to the release notes).

agreed?

[ddbeck]

Related question: In terms of documenting bugs where the flag is already flipped to true by default (e.g. feature is enabled by default), but then the flag itself is removed (i.e. you can no longer use it to disable a feature if desired), we already don't document such instances in BCD to my knowledge (although correct me if I'm wrong), and I think we also shouldn't document these anywhere on MDN anymore (I have been adding them to the release notes).

If I understand you correctly, @chrisdavidmills, I think this policy does impact some existing data. We'd need to update cases like:

• Data that concerns post-default removal of a flag. For example, a feature is added with a flag in version 38, switched on by default in version 41, and the flag is removed in version 51. The version removed value would need to be revised to 41 in an example like this. Search for layout.css.vertical-text.enabled for examples. I think this is what you're talking about?

• Data that concerns on-by-default features that still have a working flag even though the feature is on by default. The version removed value would need to be added in a case like this. Search for dom.animations-api.core.enabled for examples.

I've never personally seen a case where we have, for example, added a note to an on-by-default entry and duplicated that note on an active flagged entry (i.e., where neither entry has a version removed value).

[chrisdavidmills]

Data that concerns post-default removal of a flag. For example, a feature is added with a flag in version 38, switched on by default in version 41, and the flag is removed in version 51. The version removed value would need to be revised to 41 in an example like this. Search for layout.css.vertical-text.enabled for examples. I think this is what you're talking about?

OK, sort of. So in this example:

https://github.com/mdn/browser-compat-data/blob/e47b30524457883b6ce2bf65147933ef35f3e02c/css/properties/border-inline-start.json#L16

Yes, we'd simply need to add "version_removed": "41" to this bit of compat data, to make it correct as per your plan.

But what I'm saying/querying is that we wouldn't need to record anywhere that the actual flag itself is removed in version 51. Not many people are going to care about data on when the feature is hidden behind a flag (especially not after it has been enabled by default), so even less people will care about when the actual flag has been removed.

[ddbeck]

But what I'm saying/querying is that we wouldn't need to record anywhere that the actual flag itself is removed in version 51.

Yes, I agree with that. I don't think the removal of an on-by-default flag is worth tracking. Or rather, it might be interesting to someone, but not us as it's sort of the exact opposite browser compatibility: it's information about how to turn _off_ a feature that's supported, rather than information about how to get support for that feature.

[foolip]

My 2c on this as a more recent contributor. I think the suggestion in https://github.com/mdn/browser-compat-data/issues/3318#issuecomment-460185392 is OK, but it doesn't address my main frustration with the flag data. In reviewing and updating BCD, I'm frequently finding that the flag data is a burden:

• Considering if notes should be duplicated for both flag and non-flag entries.
• Having to update the flag entry if an error in version_added for the main entry is discovered.
• Needing to know which Chromium browsers use flags (not Samsung Internet and WebView) to mirror the right data. Even if the mirroring script handled this, it's still something one needs to understand for the difference to not be a mystery or seeming mistake.
• Sometimes having to research when flags were added just to fix incorrect non-flagged data, as in https://github.com/mdn/browser-compat-data/pull/6655.

As a reader of MDN, as I really don't like clicking the little reveal arrow only to learn that there was a flag a long time ago, and not something useful like a prefixed variant or partial implementation due to a bug that was fixed.

I would really like a policy where it's OK to remove flag data for features shipping in stable when it gets even slightly in the way, as there's very little value to the data going forward but a cost to maintaining it.

[ddbeck]

@foolip thank you for raising some specific problems you're facing. That's quite helpful. Some questions and comments:

• Considering if notes should be duplicated for both flag and non-flag entries.

Do you have an example of this? I'm not sure that I understand what this means.

• Having to update the flag entry if an error in version_added for the main entry is discovered.

Again, do you have an example of this? But if understand this correctly, does requiring a "version_removed" on a flagged entry when the feature becomes enabled-by-default not help with this?

• Needing to know which Chromium browsers use flags (not Samsung Internet and WebView) to mirror the right data. Even if the mirroring script handled this, it's still something one needs to understand for the difference to not be a mystery or seeming mistake.

This is a good point. I've opened #6662 to help address this.

• Sometimes having to research when flags were added just to fix incorrect non-flagged data, as in #6655.

That's kind of a big PR, but if I've read it correctly _all_ the data changed there is flagged. Could you elaborate more on the problem?

In general, I do want to put forward an idea that might be more immediately useful (in the absence of a removal policy for flagged data:

Faithfully communicating support stories to developers is key. If you've found evidence that some data is faulty (e.g., "version_added" is incorrect for a flagged entry), then I would prefer we regress that data (e.g., set "version_added": true), even if you don't have time to find out what's perfectly complete (e.g., set "version_added" to some accurate version number).

If you have evidence that something is wrong, but incomplete information about what's right, then I think it's a good outcome to introduce uncertainty into our data set where there is uncertainty. (Though I acknowledge that this is a regression in statistical terms.) I would merge such PRs.

In the longer term, I'm willing to entertain the idea of removing historic flag data, but so far nobody has put forward any sort of plan for making that work doable and consistent.

[foolip]

@foolip thank you for raising some specific problems you're facing. That's quite helpful. Some questions and comments:

• Considering if notes should be duplicated for both flag and non-flag entries.

Do you have an example of this? I'm not sure that I understand what this means.

This was while working on https://github.com/mdn/browser-compat-data/pull/6658. In MediaDevices.json there are notes about navigator.webkitGetUserMedia on the entry with flags, which is a useful note, which doesn't really have anything to do with the flag. Duplicating it wouldn't be ideal though.

• Having to update the flag entry if an error in version_added for the main entry is discovered.

Again, do you have an example of this? But if understand this correctly, does requiring a "version_removed" on a flagged entry when the feature becomes enabled-by-default not help with this?

It's a trivial case, but in https://github.com/mdn/browser-compat-data/pull/6658 both entries needed updating. If I had left the flag entry as "52" I don't think any lint rule could have detected my error/laziness.

Something quite like this also happened in https://github.com/mdn/browser-compat-data/pull/6655/files#diff-4ed94bdd22f3c4c82b670994673a0f19, where Opera incorrectly had version_added true. Instead of just setting it to false I went about creating flag data.

• Needing to know which Chromium browsers use flags (not Samsung Internet and WebView) to mirror the right data. Even if the mirroring script handled this, it's still something one needs to understand for the difference to not be a mystery or seeming mistake.

This is a good point. I've opened #6662 to help address this.

Thanks!

• Sometimes having to research when flags were added just to fix incorrect non-flagged data, as in #6655.

That's kind of a big PR, but if I've read it correctly _all_ the data changed there is flagged. Could you elaborate more on the problem?

Indeed, the APIs never shipped at all. Representing that correctly with flags for all Chromium-based browsers is a lot to prepare and review. If there was no flag data there already, I certainly wouldn't have added it. But this is more a complaint about the existence of flags at all, not about getting rid of old flags.

[foolip]

And now for the second half :)

Faithfully communicating support stories to developers is key. If you've found evidence that some data is faulty (e.g., "version_added" is incorrect for a flagged entry), then I would prefer we regress that data (e.g., set "version_added": true), even if you don't have time to find out what's perfectly complete (e.g., set "version_added" to some accurate version number).

If you have evidence that something is wrong, but incomplete information about what's right, then I think it's a good outcome to introduce uncertainty into our data set where there is uncertainty. (Though I acknowledge that this is a regression in statistical terms.) I would merge such PRs.

I can figure out when flags were added and enabled by default at least for Chromium, that's not a problem, although time consuming. But when I do this, more often than not I find that the flag is incompletely represented in BCD, either missing from some entries, or not correctly mirrored to some Chromium-based browsers. Especially for old flags I'd prefer to resolve such situations by just removing the flag data, but that's not currently allowed.

In the longer term, I'm willing to entertain the idea of removing historic flag data, but so far nobody has put forward any sort of plan for making that work doable and consistent.

Can we have something in the style of https://github.com/mdn/browser-compat-data/blob/master/docs/data-guidelines.md#removal-of-irrelevant-features, where removing historic flag data is allowed under certain conditions, but we don't have a process for proactively doing it?

Alternatively, if the rule was to remove all flag data older than N releases or M months/years, then writing a script to list those would not be hard. Would that help progress this issue?

[foolip]

There's discussion between @ddbeck and me in https://github.com/mdn/browser-compat-data/pull/6658 that might be worth reading. I think it's a pretty clear case where removing flag data is reasonable, but is not supported by any policy. Here's the situation:

• api.MediaDevices.getUserMedia and api.Navigator.getUserMedia were behind the same flag in Chromium, and thus shipped at the same time.
• Only api.MediaDevices.getUserMedia had the flag data, but importantly had other errors that I wanted to fix.
• Consistently representing the flag data would require adding it to api.Navigator.getUserMedia, but this would have required checked that this API was added behind a flag at the same time as api.MediaDevices.getUserMedia.

I would argue that it's not a good use of anyone's time to propose or review the flag data for api.Navigator.getUserMedia. And yet the lack of it will make it look like there's something different about these APIs that could explain the inconsistency. Making it consistent by removing the old flag data is reasonable in this case, IMHO.

[ddbeck]

Yeah, it seems like we should have an escape hatch to remove data that's historic and difficult to verify. I've opened #6670 to propose a narrow policy which should support the removal of data in circumstances like this.