Webrtc-pc: Meta: auto-publish changes to the spec

[foolip]

Over and over again, I and others struggling with the fact that https://w3c.github.io/webrtc-pc/ does not reflect the latest changes, and is now almost two months out of date.

Examples of confusion:

• https://github.com/w3c/webrtc-pc/issues/816#issuecomment-251916135
• https://bugzilla.mozilla.org/show_bug.cgi?id=1316320#c3 (@jan-ivar)

Almost all specs these days auto-publish changes somewhere. There is https://rawgit.com/w3c/webrtc-pc/master/webrtc.html, but this is not the copy anyone links to and not one you'll find via search engines.

I understand there are some reasons for the current setup. If those are enumerated, perhaps a setup that doesn't have these downsides can be found?

[henbos]

When I've linked the rawgit version in my code changes I've been told to link to the github.io version of specs instead because, I think, they're more stable/vetted, prone to contain less errors(?). Also the rawgit version has incorrect date for reference. But a lot of the times someone is implementing something fresh which hasn't reached github.io yet, so if you don't know this you're likely to get confused. If people clarify this in their changes they could tell reviewers to look at the rawgit version, but they might also be confused (took me a while before I realized there are multiple versions). I don't think everyone knows about the different spec versions, and the rawgit version is not listed at the top of the spec docs. When I'm reviewing code I usually start with the github.io version but if something seems off and I remember to I look at the rawgit too. I don't expect most people do this.

[alvestrand]

tagging @fluffy for the cogent arguments for having a reviewed editors' draft

[RByers]

AFAIK Almost all other W3C groups use their github.io links as tip-of-tree, so it's definitely surprising to implementors like me that this group behaves differently. If this group wants to follow a different process of discouraging references to the tip-of-tree specs, then I'd suggest disabling github.io entirely (relying on w3.org/TR links?) to make that clear and avoid this confusion. It'll then be obvious to implementations that tend to prefer references only to living standards (like blink) that they need to choose between unofficial rawgit links and "curated" TR links.

Note that if you decide to switch to match other groups, it's now really easy to do that. The Github repo settings now let you specify "master" as your Github Pages branch (so you can delete the gh-pages branch entirely).

/cc @plehegar who may have suggestions from a W3C process perspective.

[aboba]

With the recent move to monthly virtual interims it seems possible (and desirable) to make the tip-of-tree spec referenceable. A major obstacle is the recent instability in respec that causes the current spec to show 78 respec errors. This could be remedied either by improving the respec build validation process, or by referencing a "known good" version of respec-w3c-common.js.

[foolip]

When I maintained WebVTT using ReSpec I did just that, locked down a specific version (using Git submodules...) and only upgraded when I was in the mood for trouble.

[alvestrand]

We'll put a rawgit pointer in the README as a first measure.
We'll have to do a little more work to get procedures right (rephrase the changelog, for instance).

[aboba]

Reminder: now using respec version 8.7.1 (please note in the README file)

[fluffy]

Ok, if people want to change this, lets do it. But here is the requirement for Cisco legal. We have been engaged in litigation over IPR related to WebRTC. The assumption of juries is that a document is published with your name on the front pages of it, that you have read it. So what we can do is (and is done on many other projects) is simply make sure that any change to master is done with a PR, and all PR are reviewed by all the documents authors before being merged. This does not mean the authors need to agree with the PR, they need to have read and reviewed it so any legal issue arriving can be resolved or identified. Some people object to this, but I think that if I published something you totally disagreed with with your name on th front of it and implied to others that you had written that, well that is more or less the "fake news" problem and W3C should not do that.

I will note as a side note, that if you end up with a new version of a document ever day, you tend to not have logical points to review it. And you end up with less review. This document already receives too little review.

[foolip]

The problem of review is a serious one, but is it really helped by batching up changes and reviewing them at intervals? If I were an editor in such a process, I imagine I'd put off all review until the joint reviewing session, and once there I'd not be inclined to ask for big changes, because by then I'd already feel bad and not want to delay any further, and I also wouldn't want to waste the time of my co-editors. (Not entirely hypothetical, I have been in similar situations.)

How about just requiring review (and tests, please?) before merging any changes at all? If anyone needs to read all changes for legal reasons but wouldn't do the actual review, maybe take a daily run through open PRs and just comment with a boilerplate "I have read this" comment?

[fluffy]

Yah, on the quick review for legal reasons, I've committed to always have that done within 24 hours and given pace of standards as a whole, I don't think that slows things down.

[foolip]

Would that work then, to fully review changes before merging, and waiting for @fluffy to sign off on everything even if just skimming?

[foolip]

What are the next steps here?

This caused trouble for me again today, when looking for spec language referred to in Intent to Remove: The RTCConfiguration rtcpMuxPolicy of "negotiate", which isn't in https://w3c.github.io/webrtc-pc/ yet.

[tobie]

So this is super confusing. You're basically turning EDs into WDs and creating a new tip of branch concept that behaves as an ED does. I had to do quite some digging to figure out how to reference the fix for heycam/webidl#340.

I have no idea how this solves your legal problems, btw, as technically your name is still on a document, just not the same one, and people link to _that_ document instead.

Have you considered one of the following options that allows you to keep document terminology that matches the rest of how W3C works:

• don't merge stuff until you've actually read it (that is, add a review process or what not),
• remove your names from the ED, or
• add a big disclaimer to the ED?

[foolip]

What is the next step here? Can I do anything to help make progress?

[fluffy]

I would prefer to see us move to a process where editors reviewed PR with the github approve tool and once it was approved by editors it was merged and auto published. The current process is a hold over from before we moved to github or had ability to auto publish.

[tobie]

So there's always the option of fixing this in Specref directly. Like that, specs will link to https://rawgit.com/w3c/webrtc-pc/master/webrtc.html instead of https://w3c.github.io/webrtc-pc/ and this issue will be mostly resolved for outsiders. But I'd rather see this solved here, frankly.

[tobie]

The assumption of juries is that a document is published with your name on the front pages of it, that you have read it.

Have you considered just removing the names of the editors from the ED?

[fluffy]

Yes, that was considered but folks did not like that.

[foolip]

Is anyone actually opposed to fixing this? It's just a matter of automatically keeping gh-pages in sync right? Working around it by linking to rawgit is kind of ugly, I don't want to go changing the URLs for this spec everywhere I find them.

[foolip]

@sideshowbarker, can you comment on this? What is your recommendation to WGs for what github.io should reflect?

[sideshowbarker]

@sideshowbarker, can you comment on this? What is your recommendation to WGs for what github.io should reflect?

My recommendations in general are:

• Absolutely nobody taking time to review the spec and file issues against it ends up wasting their time reviewing a stale, out of date copy somewhere
• Even more importantly, for any specification that’s intended to be implemented, great care should be taken to ensure that no implementor ends up working from a stale, out of date copy

We have many years of evidence implementors and other contributors having their time wasted and being frustrated due to having used a stale, out-of-date spec version — in cases where WGs have not cared enough to try to make sure that doesn’t happen.

I know nothing about the specifics of the case being discussed in this issue, so I dunno, the above may actually be completely irrelevant. If so, then just ignore this. But if what’s being discussed there is in fact a case that risks causing implementors and reviewers to work from a stale, out-of-date spec version, the please do consider it carefully.

[fluffy]

As I have said before, I would love to see this move to what I view as a normal process of a PR is made, the editors all approve the PR using the github approve stuff, and then it is merged to master and auto published.

That said, I do not see us getting a lot of out of date reviews. The best version to review is https://www.w3.org/TR/webrtc/ as it is a consistent set of functionally. If you are reviewing or using something other than that, we tend to get reviews on stuff that may not be ready as changes are still in flight.

[foolip]

@fluffy does this mean that you wouldn't mind if https://w3c.github.io/webrtc-pc/ is auto-published, since https://www.w3.org/TR/webrtc/ is the version you prefer to look at anyway?

[fluffy]

I would be OK with

I would love to see this move to what I view as a normal process of a PR is made, the editors all approve the PR using the github approve stuff, and then it is merged to master and auto published.

I'm not OK with other things.

[tobie]

I would love to see this move to what I view as a normal process of a PR is made, the editors all approve the PR using the github approve stuff, and then it is merged to master and auto published.

It's increasingly common to have a normal process include automatic push to TR as WG drafts.

[dontcallmedom]

for clarity, we already have auto-publication from gh-pages to TR. Our oddity is that we have master irregularly sync'd with gh-pages.

[foolip]

@fluffy, if you want to require review from all editors, I think you can have tooling for that, or just a convention that everyone with merge powers agrees to. Do you think that auto-publishing should be blocked on that though? If so I expect to see no commits to this repo until this issue is resolved, which would be fine with me.

[foolip]

@dontcallmedom, sounds like the fix is to instead have a tr branch that's auto-published to TR, and to have master auto-publish to github.io. Would that work?

[tobie]

Or even auto-publish the ED and stop auto-publishing to TR.

[dontcallmedom]

@foolip @tobie based on what I understand from @fluffy that would not address his concern.

I'll be discussing with the chairs how to fix this trying to take into account the various constraints - should happen next Tuesday.

[foolip]

OK, so the only thing I'm arguing for (care about) here is that the version of the spec that we link to everywhere, https://w3c.github.io/webrtc-pc/, reflects what is on the master branch of this repo, so that when people see that a PR is merged, they can actually go read the spec there.

TR I'll leave to someone else to have an opinion about.

[fluffy]

I'm not looking for any tool level enforcement (although that would be nice) I am just looking that I get a chance to review the PR before it is published for certain issues related to a ongoing patent lawsuit. I am not asking that I agree with the PR in anyway, if the chairs believes the PR is the right thing to merge that is totally fine with me but I don't want it merged without the chance to review it and have a discussion with chairs and other editors in the extremely rare case that the PR has an impact for the community related to theses legal issues. (and to be clear on which side we are on here - we are defending against other who are alleging we infringe their patents. I believe that if courts agree with our position that it will help other users of this technology not have to deal with theses patent claims). I'm happy to explain in person more details on why this scan for particular legal issues is needed. It happens really quickly and is not a review of 99% of technical content that is in most PRs.

[tobie]

Why can't this review happen before PRs are merged?

[foolip]

@fluffy, does the distinction between https://w3c.github.io/webrtc-pc/ and https://rawgit.com/w3c/webrtc-pc/master/webrtc.html matter for legal purposes? Isn't it sufficient if https://www.w3.org/TR/webrtc/ is blocked on your review?

[foolip]

@alvestrand, if this can't be resolved in a GitHub issue, would you mind adding it to the agenda for the next VI?

[dontcallmedom]

@foolip I have tentative agreement to switch to a model that hopefully will work better for everyone:

• instead of reviewing complete editor drafts, we would ask @fluffy to review pull requests prior to their merging
• once that review is done, merging pull requests will directly lead to updated editors draft

I still need to figure out some of the details of the implementation, but hopefully we're on the right track

[foolip]

That sounds great, looking forward to it! Is there a backup for @fluffy if he is on vacation or otherwise not able to review PRs on a day-to-day basis?

[fluffy]

Yes pretty much anyone that deals with legal stuff at Cisco - glad to add people from Google legal if they want to be involved with analysis of patents that may apply to hangouts.

[dontcallmedom]

http://w3c.github.io/webrtc-pc/ reflects now top of the tree in master. gh-pages will go away once we're confident the new workflow works as expected.

[foolip]

Yay, thanks @dontcallmedom for making this happen!

[alvestrand]

Everything seems to be working. @dontcallmedom can you close the bug once you've deleted the gh-pages branch for good?

[dontcallmedom]

gh-pages deleted, closing.