Gutenberg: Collaborative editing using WebRTC

[ehg]

Collaborative editing in WordPress has been on our minds for a while. What we've been lacking is a way to support this on self-hosted sites, without the need for a 'centralised' service, whilst retaining compatibility with hosts who may not have the ability to dedicate long-running processes that lends itself to most collaborative editing techniques.

I proposed an experiment, to @abhishekgahlot — what if we were able to use WebRTC, with the combination of WP-API endpoints, to accomplish this goal?

The original plan was to go full-on Google Docs style collaboration, with p2p Operational Transforms, or CRDTs. We then took a step back, and thought, due to the nature of blocks — how about just using traditional locking, similar to the current WordPress editor, but _per block_. We decided this should be enough granularity for now, and will get us to a proof of concept a lot faster.

If this proof-of-concept is feasible, we can of course implement "full" collaborative editing at a later date. It also opens up the possibility of audio/video/text chat, if we ever see a need.

There are some things we need to watch out for when considering the implementation:

• Latency – is WebRTC going to be fast enough?
• Scalability – how many editors can we reasonably support?
• Reliability – what happens on dodgy WiFi, or if a user disconnects etc.
• Compatibility – WebRTC isn't supported on IE11, and some older browsers. Also, how do we get around firewalls?

This issue is to track the various approaches we're trying out, and will be updated as we progress in our experimentation. For now, a pull request is available at https://github.com/WordPress/gutenberg/pull/1877, including more specific design details. Thanks @abhishekgahlot :)

Here's a preview of the proposed initial design, kindly provided by @jasmussen:

[westonruter]

Post locking in WordPress currently has a 150 second window, so WebRTC will surely have a double-magnitude reduction in latency compared to that. Similarly, it will have a magnitude reduction in latency compared to WP Heartbeat, which itself is often disabled by hosts. That's the beauty of WebRTC: there aren't hosting requirements… mostly.

I assume this would mean WordPress would need to include a STUN and TURN functionality?

[abhishekgahlot]

@westonruter Yes, We will be using STUN/TURN ( Hole punching and for relaying ) also all the network traffic will be P2P and encrypted. Signaling will be either XHR or WebSocket based.

[westonruter]

@abhishekgahlot but how much of this will be able to run on the server with WordPress itself? Will there be a need for an external service? For example, most hosts wouldn't support WebSockets. It would be great (required?) if it could be self-contained.

[abhishekgahlot]

@westonruter As of now, everything works standalone without the need of WebSockets for 2 peers. Because signaling is done by XHR and number of requests to verify how many peers are there is less. What I meant was we can utilize WebSockets for faster initial signaling process.

[ehg]

For example, most hosts wouldn't support WebSockets. It would be great (required?) if it could be self-contained.

Yes, this would be one of the constraints. We need to go as far as possible for it to be standalone, and not rely on any centralised service, hence the initial idea of WebRTC + OT/CRDT in the first place.

[jasmussen]

Love this ticket.

I would attach just a quick note, that it's important to only focus on the colored block outlines in this post, and imagine them in the current design — that is, the 2px border with a color and a name attached, rather than a full width dotted line. The mockup is very very old and discarded for many reasons. Though I do miss the Spinal Tap boilerplate text ;)

[jasmussen]

Here's a new mockup. CC: @abhishekgahlot and #1877:

[westonruter]

@jasmussen should the block be read-only for user B when user A is editing it? When it has the presence indicator, would it be locked?

[jasmussen]

@jasmussen should the block be read-only for user B when user A is editing it? When it has the presence indicator, would it be locked?

Yes, in the mockup above, I'm editing the first paragraph, and Abhishek is editing the 3rd paragraph. The 3rd paragraph is locked to me, and the first paragraph is locked to Abhishek.

[westonruter]

@jasmussen and how would the locked state manifest itself? None of the controls would appear when you select it? Maybe it should have a message overlay like “This block is currently being edited by X” to make the locked state explicit? Otherwise a user could be confused as to why they can't edit the block, since Google Docs allows concurrently editing the same “blocks” (paragraphs).

[jasmussen]

and how would the locked state manifest itself? None of the controls would appear when you select it?

That's been the idea so far, yes.

Maybe it should have a message overlay like “This block is currently being edited by X” to make the locked state explicit? Otherwise a user could be confused as to why they can't edit the block, since Google Docs allows concurrently editing the same “blocks” (paragraphs).

This is a really good point. I like the more verbose "Being edited by [username]" label. We might even show a graying overlay on a locked block, perhaps even change the cursor from the caret to a stop sign when you mouse over a locked block. I think some of this can be worked out in implementation, but these are good ideas.

[jasmussen]

When it becomes time to color the various borders of blocks locked to others, we shouldn't use arbitrary colors. We should use from a preset array. Here are some colors I'd suggest using:

red: #dc3232
orange: #f56e28
yellow: #ffb900
green: #46b450
purple: #826eb4

These colors are from @hugobaeta's extended WordPress color palette: https://codepen.io/hugobaeta/full/RNOzoV/

If more than 6 people are editing the same post, we can just repeat the colors — there's also a name attached to each block anyway. Or we can double the amount of colors by also using a lighter version of the same shade.

We can use these colors by adding them to classnames, and then simply apply a CSS class to a block that's locked. CC: @abhishekgahlot

[rheinardkorf]

Is there a prototype for this issue yet?

I'm currently working on a scenario where multiple repeatable editors are required. In most cases this is not a big concern is there is only one editor. But, multiple users may need to collaborate. Would be nice to start playing around to get a feel for how this could work.

EDIT: Never mind, just saw the working PR for this.

[gziolo]

Yes, there is one as you already discovered: https://github.com/WordPress/gutenberg/pull/1877. I’m linking to make sure everyone else is also able to easily locate this in progress PR.

[gziolo]

@jasmussen can we include in the mockup a list of all users editing the post/page. @mtias suggested to show connected authors in the top left.

[jasmussen]

I'm a bit weary of putting stuff inside the main editor canvas. If we hope to load the theme stylesheet there in the future, we can't know for sure which kinds of crazy backgrounds floating items will sit on (see also my comment on the recent Word Count feature).

In this case, seeing the active editors, it feels like we should emulate the Docs pattern. It also feels like this is a feature that's okay to completely hide on the mobile breakpoint.

In the following mockup, the toolbar is slightly rearranged and tweaked (see #2460), freeing up space to show little avatars next to the save indicator:

CC: @karmatosed

[gziolo]

In this case, seeing the active editors, it feels like we should emulate the Docs pattern. It also feels like this is a feature that's okay to completely hide on the mobile breakpoint.

This is how I pictured it myself, too 👍

[jeffpaul]

_This ticket was mentioned in Slack in #core-editor by jeffpaul. View the logs._

[Lewiscowles1986]

Hello, I've been meaning to check this out since @Karmatosed mentioned at WordCamp London.

My concerns are

• WordPress meta and posts seem to be "document-oriented" in their relationship
  
  
  
  • how to deal with race conditions if person {A} updates block {X} when person {B} updates block {Y} on the same post.
  
  
  • Are blocks already stored separately? In my wp install it seems to all be in the post_content db field.
  
  
  • is it okay that neither request can reasonably know about eachother, so resolution of races might be problematic
  
  
• Technical complexity of deploying a STUN / TURN server that is turn-key and safe

It looks AMAZING, and I really want this for WordPress, because it's amazing. But like the initial TinyMCE wp.views, I feel like there won't be significant enough a portion of people that use or know about this to be able to contribute, or extend into their own plugins.

[gziolo]

Technical complexity of deploying a STUN / TURN server that is turn-key and safe

In the proof of concept we have, there is no server involved. It works for 2 peers and one of them plays the role of the server to make sure that it can take precedence when resolving conflicts. They communicate between each other using WebRTC.

Are blocks already stored separately? In my wp install it seems to all be in the post_content db field.

Inside Gutenberg all blocks are stored independently in the memory, so we were able to lock only those blocks that are currently being edited. It needs some more work to make sure it works with multi select or drag&drop but those are issues that can be resolved with good design decisions.

how to deal with race conditions if person {A} updates block {X} when person {B} updates block {Y} on the same post.

We replay the same (Redux) actions on each client to make sure that their internal in-memory state is in sync.

[Lewiscowles1986]

Thanks, so it's peer-to-peer in the proof of concept.

[gziolo]

Thanks, so it's peer-to-peer in the proof of concept.

Yes, this is what it is. It only makes sense in WP core when it doesn't require additional setup and external resources. We can make it extensible so plugin developers could make it work for more peers using the STUN / TURN server.

[Lewiscowles1986]

The idea I had upon reading would be that like multisite and other features WP_DEBUG you'd probably use defines and if they were not present, you'd enable shared hosting to just work without (since it likely would have too much latency for shared hosting to be effective anyway).

The concern over STUN/TURN if it were a thing would be hackers getting into the servers, either leeching for their own services, or maliciously using them for attacks like unprotected memcache server daemon.

Anyway. Thanks for PoC, very interesting idea and choices. :+1:

[stebansaa]

What is the status of this feature?

[gziolo]

It isn't actively worked on nor explored. It won't be a priority until Phase 2 of Gutenberg is close to beeing finished.

[gziolo]

I’m sharing a link to an article from The New York Times for a future reference:
We Built Collaborative Editing for Our Newsroom’s CMS. Here’s how.

[dmonad]

We are currently exploring ways to make Gutenberg collaborative with Yjs. Here's a great blog-post explaining our goals https://publishpress.com/blog/yjs/

We are pushing updates on our work in #18357