Slate: Collaborative editing

[davolokh]

@yatskevich and me did a research and tried to design an appropriate architecture for supporting collaborative editing. There are some key points with explanation below.

The main idea is to implement a kind of Redux-like behavior within Slate.
<Editor store={Store} … />
The goal of having Store is ability to have clear separation and understandable flow.

UseCase:

• User press key (types symbol)
• Action is generated
• Action is dispatched to Store
• Store updates State
• setState called
• Editor re-rendered

This way allows us to have ability to manage operations in any suitable way (CRDT, OT, etc.) within Store.

Let’s see how to handle text paste operation in more details:

1. Action pasteText dispatched to Store (data: {position, text})
   No need to get current state, call transform() and return new state. Just dispatch an Action.
2. Store has a Reducer for handling such an Action
   If we’re talking about ColloborativeStore - one type, if about SimpleStore - another one. We can even have few implementation of CollaborativeStores (CRDT and OT).

CollaborativeStore:

• Splits pasteText operation into a set of atomic insertCharacter operations
• Applies operations to State
• Translates this operations set to server
• Listens for updates from server (i.e. smb else is editing this doc and generates the same type of operations set)

SimpleStore:

• Applies operations to State (no need to split to atomic operations)

In our mind such architecture allows Slate to be more flexible and extensible. But this is not a completely analyzed proposal - just an idea.

And a few pictures for clarification.

Workflow Chart:

Implementation example

pastText in depth:

P.S. Possibly, Store should send Transfroms (not Actions) to Server and get Transforms back.

@ianstormtaylor: Could you please take a look at this and share your opinion?

[ianstormtaylor]

Hey @davolokh thank you so much for writing this up!

I'm definitely interested in figuring out the best way to make Slate collaboration-ready. I've been doing a bunch of research on OT/CRDT design to get the best solution. And if there's a way that the Slate core can remain unopinionated about OT vs. CRDT that would be awesome.

From all of my research it seems like OT is slightly more complex to implement, but is much more real-world-proven since it is what Google Docs, Dropbox Paper, etc. use to implement their collaborative editors. So I'm focusing on its needs first, which seem to overlap with CRDT a lot anyways.

It seems like what we need for the two is this:

• OT
  
  
  
  • [x] Transform needs to be composed of smaller, simpler "operations".
  
  
  • [x] State changes need to expose the operations that occurred.
  
  
  • [x] Operations need to be invertible, for undo history.
  
  
• CRDT
  
  
  
  • [x] All of the same needs as OT, plus...
  
  
  • [ ] Character-level unique keys.
  
  
  • [ ] A way to "tombstone" keyed objects.
  
  

(If anyone has additional requirements for either OT or CRDT that I need to add to that list and be aware of let me know!!)

Once we have those things, it seems like OT/CRDT can be implemented alongside Slate. But those things also aren't dependent on the core library changing much.

As for migrating it all to a Redux system, I'm slightly against it because it seems like a lot of overhead that makes Slate more confusing to people who aren't Redux-fluent, and it seems like it goes against how most React components are built. (For Hyperterm it actually makes a lot of sense, since they are essentially trying to make an entire app pluggable.)

I _think_ that the current state can be changed to allow for "operations", by just exposing a single extra argument to onChange handlers. Taking us from the current:

function onChange(state) {
  // apply the change locally by setting new `state` value
}

To:

function onChange(state, change) {
  // apply the change locally by setting new `state` value
  // send the `change.operations` to the server
}

Which is passed a transform argument which describes the transforms and operations that took place to change the state. (I'd actually like to rename this object from Transform to Change to make it more clear, and to remove the confusion with OT's definition for "transforms".)

And then the plugin handlers like onKeyDown or onPaste would simply return a Change object instead of an entirely new state, which is the equivalent of removing the .apply() call. Taking us from:

function onKeyDown(e, data, state, editor) {
  // return a new `state` object by transforming the current one
  return state
    .transform()
    .insertText('some new string')
    .deleteForward()
    .apply()
}

To:

function onKeyDown(e, data, state, editor) {
  // return a `change` object by running transforms on the current `state`
  return state
    .change()
    .insertText('some new string')
    .deleteForward()
}

Let me know how that sounds! I'm still on vacation for a few more days, but hoping to get moving on this again next week.

Cc @SamyPesse too in case he has good thoughts, or in case I'm forgetting something obvious.

[kommen]

Hey there, just throwing this in as I'm following this issue here and it seems relevant as your deciding between CRTD/OT: http://arxiv.org/abs/1608.03960

I'm just reading/digesting the paper and eagerly waiting for the reference implementation to be released. We'd probably try to use it with the Store interface you are coming up with.

[takashi]

Is there any progress about implementation of OT/CRDT on Slate? I'd just like to know about.

[CooperSystems]

Would be great to expose the 'change(s)' in onChange and/or get any way to get a list of changes in order to sync with other clients.

Currently looking at syncing entire state between clients, with logic to throttle and ignore changes by the current clientID, which seems hacky

[ghost]

Real time collaboration would certainly be a great win for SlateJS. I'm sure you are aware of how ProseMirror is implementing it, which looks like an approach that should also be working with Slate in general:

http://prosemirror.net/guide/collab.html
http://prosemirror.net/guide/transform.html#rebasing
https://github.com/ProseMirror/website/tree/master/src/demo/collab

But I also wanted to point you to another great project by @dmonad called Yjs:

https://github.com/y-js/yjs

It is a general purpose framework for real time communication, with sample implementations for QuillJS and other text inputs. It provides some abstracted data types, which might be a good basis for additional communication.

[kminehart]

@ianstormtaylor I'm curious if there's a branch where you've got these changes running so far. I'd love to contribute; getting Slate to work collaboratively is something we're very interested in.

[alanchrt]

This is somewhat a note to my future self (I don't have the time to invest right now, hopefully soon), but also for visibility and a vote for keeping Slate unopinionated about OT vs CRDT.

I'd like to take a stab at a plugin at some point to translate Slate transforms into the OT operations listed here:

https://github.com/ottypes/json0#summary-of-operations

This would make Slate play nicely with ShareDB and hopefully other collaborative OT stacks.

It would probably require a lot of the work for change hooks/events/whatever discussed in this issue to be knocked out first to get at the proper transform operations, but hopefully not too painful after that point.

I'm just getting my head around OT/CRDT, so I really don't actually understand the implications yet or how much it would be possible to stay decoupled, but +1 for just exposing the raw transform/change data and letting plugins handle the rest.

[olivoil]

Is there any progress made on this? just curious to see if anything was started...

[alanchrt]

Nothing on my end! Still seeming pretty far out from having a window of time for it.

_Wanting_ to do something is just as good as doing it.. right??! :bowtie:

[trungtin]

Will something like this help:
https://github.com/google/ot-crdt-papers (author post https://medium.com/@raphlinus/towards-a-unified-theory-of-operational-transformation-and-crdt-70485876f72f)

I will try to understand both this theory and how Slate work under the hood first 😄

[aslakhellesoy]

How about using crjdt?

[akashkamboj]

hi @ianstormtaylor I studied mostly all of the editors out there and everyone has some issues. I was working in draft-js until I reached to the stage of collaborative requirements. tried doing some hacks into draft-js, by calculating diff-patch and reach to a stage but as soon as undo/redo stack part came, everything failed. Came to your repository, wish I found it earlier and I liked the way you have structured the project and easy to understand coding. This time I am trying the collab part first and the changes you mentioned in your comment above seems a good start to try the hands on. Are you working on these? Do you have any timeline for this. Any way I can contribute to this to make things faster.

Edit: also one more question? Is there a way to do transform without inserting into undo stack currently? or that's why you are suggesting change/transform separately. : Found it

[ianstormtaylor]

Hey all, thanks for the interest in this issue! To answer some of your questions...

I definitely still want this to happen. Slate core is very close, I think the last remaining change will be to make "transforms" the first-class thing that plugins return from event handlers, instead of right now how they return full-fledged State instances. (Basically you'll just omit the .transform() and .apply() from the plugin code, and they'll chain together on a single transform instead.)

I'm going to focus on OT as our collaboration algorithm, instead of CRDT, because in general it's much more proven for the kind of work we want to do, and there's a decent amount of existing work that can help—ShareDB is probably the best way to go for getting it working. That said, if you really want/need CRDT feel free to charge ahead in that direction, and I'm down to keep Slate un-opinionated regardless. But I think for almost all of the cases people have brought up so far that OT is the better fit.

The collaboration aspect will be a lot of work. Feel free to throw progress, thoughts, etc. in this issue and we can all be updated if people are working on it.

[thomsbg]

I like the direction this is headed in! I've started some preliminary exploration of a slate-ottype library in the vein of https://github.com/ottypes/docs, and wanted to float some ideas:

• The rich-text ottype isn't quite expressive enough to encode a Transform object, because it's limited to just insert, delete, and retain (format). It's fundamentally a string with attributed ranges rather than a tree structure.
• The json0 ottype isn't sufficient to granularly express the operations indicated by a Transform object. As a list of operations, however, it seems like a relevant starting point.
• Comparing the approaches taken by rich-text and json0, I quite like the "sparse traversal" format of a rich-text delta over the "list of operations" format of json0 as a way to encode a changeset. The time complexity of a "sparse traversal" should be lower than that of a "list of operations" implementation. This is noted in the json0 readme: https://github.com/ottypes/json0/blob/master/README.md#commentary.
• What might a "sparse traversal" of a tree structure look like? Might that impact the underlying design of the Transform class?
• If taking the "list of operations" approach, is seems like there needs to be a transform function for each pair of fundamental operation types. With 13 different operations (addMark, insertNode, insertText, joinNode, moveNode, removeMark, removeNode, removeText, setMark, setNode, setSelection, splitNodeAtOffset, splitNode), that's 78 different functions (13 choose 2)! Is that right? Maybe there are some shortcuts or symmetry to take advantage of.

[ianstormtaylor]

@thomsbg good insights! I totally agree. I think both rich-text and json0 don't quite support our needs, but they are good for context of what's possible. There's also a json1 that Joseph has been working on again recently (on his personal GitHub).

I'd be very curious about your (or other's) take on the sparse traversal approach, and how that might apply to our nested tree.

I think in terms of operations we can do some smart things, but not totally sure, for example for remove_node we can probably handle all cases of operations with a path by just decrementing the overlapping index. Instead of needing to handle it for each combination. Same for insert but in reverse. Of course, it'll still be pretty complex.

Anyways, great write up! Would love to hear more thoughts if you dig deeper.

[philcockfield]

Hey @ianstormtaylor - I am watching this issue/thread with eager anticipation. Are you working on this stuff on a different branch? Wondering how long it would be before enough hooks are in place to have Slate working against a ShareDB in a sample repo?

Is this a long ways away? Or something we can plan on employing relatively soon?

Thanks!

[olivoil]

Is there a consensus to use OT instead of CRDT?

If Slate used CRDT, it would allow it to sync without the need for a central authority like shareDB at all, and could just work out of the box with clients connected to each other without a server-side component. It would also make offline work and syncing much easier than with OT.

Y-js has been suggested earlier in this thread and my experience with it has been surprisingly good.

If there's still some room to influence the direction this feature is taking, I would vote for using CRDT or making it agnostic and easy to plug something like Y-js.

[ianstormtaylor]

Hey @philcockfield, I wouldn't assume a short-term OT unless people contribute more pieces that are required for OT. Even fixes bugs that would prevent OT from working (like the inline ones) would be helpful.

@olivoil I haven't seen much in terms of CRDT's actually being used in real-world scenarios, so there's a lot less prior art out there, if it's even possible. For that reason I'm personally going to opt towards OT implementation over time, it's just more proven. Using something like ShareDB would ensure that lots of the work is handled for us out of the box, which is a lot of upside.

[philcockfield]

Thank @ianstormtaylor.

The pros of CRDT that @olivoil outlines seem very nice - but I'm guessing that the various hooks necessary to get OT operating would also work for CRDT (I say naively, not in any way actually knowing!) @olivoil, is it reasonable that the necessary hooks would be applicable to both OT and CRDT?

Leaning on more extensive prior-art with OT (with ShareDB as a example lib) makes sense to me. Thanks @ianstormtaylor - and thanks for this wonderful library.

[tvedtorama]

Thank you all for working on this extremely useful feature! I would really like something that could plug into the state manager (Redux in my case) and do the merges/transforms.

In the meantime, has anyone had any success with a poor man's version - such as (explicit) paragraph locking or other approaches that prevents concurrent updates?

[philcockfield]

@tvedtorama I too am super interested in any poor man's hacks on the way to full collaborative editing.

I've been playing with the idea of a locking blocks based on who's editing, and putting a lot of "social presence" feedback in there, namely tiny avatars on each block, so it's clear why something is locked.

I have not yet implemented it though.

Also, I echo your sentiment, thank you everyone who's been working hard on this SUPER SUPER USEFUL feature.

[TimYi]

@ianstormtaylor I have made a collaborative product, I'm the backend designer, we use ot algorithm, so I know what is needed by an ot collaborative editor.

Our current collaborative editor is a compose of ckeditor hand handsometable, customized a lot, and the code looks awkward.

I'm searching for a proper editor program to build our new collaborator editor on top of it. I looked at draft js, it's react model and immutable data type seems attractive. In theory, it's easy to diff immutable data, and get the changes, but I dig into draft and find it hard to separate the changed part from the whole state.

Then I found slate, see the documents, and I think slate is just one step away from collaborative editing, at least for ot algorithm.

I looked your answer to @davolokh, your idea is totally right, the last step is map a Transform to the change of the state. We should make use of immutable model, make proper abstract of a change. And then the upper level code can map this change to their operation model easily.

Different collaborative document model have different view of change. Google wave treat the whole document as a string, and properties like bold is anchored to the string by position. Some collaborative document use json model, and the property and texts are nodes of a tree.

So a good abstract of Change object is important for all possible collaborative model to build their own operation object.

[akashkamboj]

Here's my experience of Draft.js / Slate / Quill. Worked on all 3 of them.

Draft.js - Very nice, if you are working as a solo user, completely lacking Collaboration part and if that's your need, forget it. And it will take years because it has to be completely rewritten from core. The community is very large, you get most of the answers to your questions. Although if you need to change something in core, you will have a lot of trouble, especially selection module.

Slate - Nice, but again collaboration is missing. although seems like @ianstormtaylor is interested in implementing this, but very slow progress is going on. And not sure if Slate will achieve this before Draft.js or not. Plus I really didn't understand why everything is Operation based task, if collaboration was not the intent from the very beginning. And sometimes editor becomes very heavy, especially if you copying pasting large content blocks from somewhere. Draft.js is very fast. I hope if Slate implements Collaboration before Draft.js it will be a big hit. Community is not very big for Slate, you are your own or if you're lucky maybe Ian will answer some of your questions.

Quill - Very Very Nice but sucks at many places. I've implemented all three and finally settled down on Quill, because collaboration is there and it's almost working fine. Sometimes a few places somethings goes wrong, mostly in undo/redo. But overall it's working ok. But and the big BUT is that it's not in React so I had to create a wrapper for it in React and I can't do interesting things beyond a certain level. Plus their Block system is too confusing, because it's entirely on DOM and every minor thing that you have to do you are mostly dependent on the DOM attributes. Community is again one of the problem, it's not big.

Final Thoughts: For now Quill is working fine for me. But I would anyday switch to Slate or Draft.js if they implement Collaboration.

[tobiasandersen]

@akashkamboj Now I haven't used Quill, so I'm not going to comment on that. But I've used both Draft and Slate quite extensively, and I have to disagree on some of your points. Especially this part:

Although seems like @ianstormtaylor is interested in implementing this, but very slow progress is going on. And not sure if Slate will achieve this before Draft.js or not. Plus I really didn't understand why everything is Operation based task, if collaboration was not the intent from the very beginning.

The reason everything is based on operations (or transformations) is that @ianstormtaylor indeed did intend to support collaborative editing from the very beginning. This is key, and one of the major reasons why I moved away from Draft.

I also don't feel that the progress on collaborative editing is slow. @ianstormtaylor is constantly updating the branch on which he works on it, and people are helping him test it. To me it seems that Slate soon will have all the pieces needed in place.

I also wonder a bit about this:

Community is not very big for Slate, you are your own or if you're lucky maybe Ian will answer some of your questions.

Things might have changed since I moved from Draft to Slate, but I actually prefer the community of Slate. The Slack channel is pretty active, and people are both friendly and helpful. While this is also true for Draft, the thing I like with Slate is that @ianstormtaylor is very open to thoughts from the community.

Which brings me to the other major reason for my move to Slate — Draft, compared to Slate, is incredibly reluctant to changes in core. Not because their maintainers are bad people (they're super nice and smart), but because Draft is used in a lot of different Facebook products. So each change has to go through each of those teams before being approved. None of FBs products require collaborative editing, and until any of them do, I highly doubt that support for it will be added. And even then, as you point out, that would require big changes to core.

Sorry if this came off as confrontational @akashkamboj. You're of course entitled to your opinions and experiences. I just wanted to share my view on the same topic, thanks for sharing yours. Also, I really don't mean to shit on Draft. I enjoyed working on that too, and they have some really nice people maintaining it. But for us, Slate has been better in almost every way, so big props to @ianstormtaylor!

[philcockfield]

The Slack channel is pretty active, and people are both friendly and helpful.

@tobiasandersen where's the Slack channel?

[tobiasandersen]

@philcockfield https://slate-slack.herokuapp.com/ :)

[ianstormtaylor]

Hey all, the OT branch was merged in 0.22.0 so everything that unlocks OT should now be present in Slate's core! More specifically:

• All changes are composed of low-level operations.
• The Change objects are available for each change, with the exact operations that occurred.
• The operations contain all information and are sufficiently invertible for OT transforming.
• The apply and invert operation helpers are exposed.

I don't think Slate's core will actually add the OT logic itself, since it isn't used for all use cases and would add bloat. So instead, it would live in another package. If anyone wants to start collaborating on creating that package that would be amazing!

If you're working on that, and you run into any places that Slate's core is still missing functionality required, let me know and we can get it fixed. But I'm pretty sure it's all there.

[YurkaninRyan]

@ianstormtaylor Awesome work, huge gain for Slate!

[philcockfield]

@ianstormtaylor - man!! Thanks so much for all this inspired work. This is so massively helpful to have OT.

[tvedtorama]

If I understand this correctly, the Slate data model now have the granularity and structure to allow conflic-free edits and merges - which is excellent. For the moment, however, it's the application developer's job to do the actual merges when concurrent edits are received from peers (server signaling / webrtc). Is that correct?

Can we use existing tools for that, such as the y.js library? Or would this model require it's own implementation?

Again, thank you for this great and inspiring work!

[aslakhellesoy]

I've been playing around with Automerge lately (independently of Slate). It's a JSON CRDT that I think would be a perfect fit for Slate.

[ianstormtaylor]

@aslakhellesoy, that looks really interesting! Let us know if you get it working with Slate, seems like a very cool option.

[alanchrt]

Whoa! I've been looking for a pure javascript JSON CRDT implementation for the past couple weeks.

I'm working on an offline-first app, and that could be huge for apps like mine that need to merge after offline collaboration.

Once I get to the actually-implementing-collaboration bit, I'll be taking more of a look.

@ianstormtaylor congrats on exposing the change operations! So cool to see the progress.

[aslakhellesoy]

@ianstormtaylor I'm not currently working on Slate/Automerge integration, nor do I plan to in the near future. Maybe in the distant future.

OTOH I've just implemented CodeMirror/Automerge integration which could be a decent starting point for a Slate/Automerge plugin.

[geakstr]

Hi! I try to use latest builds and info about operations is awesome. Thank you, @ianstormtaylor, it looks like that Slate is the editor of dreams 😄

[nornagon]

If anyone's interested in working on an ottypes implementation for the slate.js model, hit me up! I'm working on an app that this would be perfect for :)

(And I've built some complex OT types before...)

[lxcid]

@nornagon awesome. I love to take part in the project!

[nornagon]

@lxcid cool! I'm hanging out on the Slack space if you want to coordinate :)

[lxcid]

I’m in there as @lxcid as well! Let’s coordinate there! What’s your nick? I should be back on my desk in 5-6 hour time. :)

Sent from my iPhone

On 27 Sep 2017, at 1:50 PM, Jeremy Apthorp [e-mail] wrote:

@lxcid cool! I'm hanging out on the Slack space if you want to coordinate :)

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[philcockfield]

Are there any updates to documentation anywhere that calls out these new OT related features in some kind of collaborative editor context? Thanks.

[mshibl]

Hello @ianstormtaylor thanks for the great work on this ...

We're currently considering switching from DraftJS to Slate .. for the specific reason of needing collaborative editing. I'm currently going through the API to see if it allows us to do what we need to do.

The question I have is, I'm now pulling out the Change object from the onChange function and sending the change.operations to other users on the page .. on the receiving end, how do I apply these operations on the current state and generate a new state accordingly?

[mshibl]

UPDATE The approach I have below is missing key pieces as I figured out after testing .. please see the comments below

Following up on my last comment ... I was able to figure out a few pieces of the puzzle, and I would like to share what I'm doing with everyone here.

I have 2 goals I'm trying to achieve:

1. all users on the page should be able to see all changes made by other users in real-time
2. as I'm editing a page, I should see the cursor location of all other users on the page

To achieve the first goal, I'm doing the following:

• Connect all users on the page through a WebSocket connection
• Every time a user makes a change I do the following:
  
  
  
  • intercept the change in the onChange function
  
  
  • pull out the operations from the Change object
  
  
  • filter out any selection changes operations (other users shouldn't care about those)
  
  
  • send the array of operations over the WebSocket to all other users on the page
  
  
  • On the receiving end. I pass the received operations to the slate instance
  
  
  • I create a new Change object from the received operations using state.change().applyOperations(operations)
  
  
  • I pass the deduced Change object to the onChange function and Voilà the changes are applied
  
  

To achieve the second goal, I intend to do the following:

• In the componentDidUpdate function, I pull out the new selection from the updated state.
• I convert the selection to raw format using the toJSON method
• I replace the focusKey and anchorKey with relative paths instead of explicit keys using the state.document.getPath(key) function. This step is important since keys are not unique identifiers and could be different from one slate instance to another
• I send the updated selection along with some user metadata (name, avatar, etc.) to all other users on the page
• On the receiving end.. I get the updated remote user selection. I decode the anchorKey and focusKey back using the state.document.assertPath(path) function to get the correct keys.
• Using a special Mark that I call UserCursor that decorates the text currently selected by the other user with a color background and adds a small tooltip above that text that shows the user's name or avatar. I add that mark to the selection range that was just received, using the function state.change().addMarkAtRange(Selection, Mark) function
• Before I do that last step I need to remove the user's existing mark (if it exists). I keep track of the Marks (UserCursors) on the page using a map of key-value pairs. The key is a unique identifier of the remote user (ex. user_id) and the value is the key of the current block that holds the Mark. Using this combination I can remove the existing UserCursor using the state.change().removeMarkByKey() function.

---

As of now, I was able to finish 80% of the first goal, and it seems to be working well so far. However, I thought it would be a good idea to share my approach with anyone interested. I'm also interested in any feedback I can get.

@ianstormtaylor I'm also interested to know why some very useful functions like assertPath, getPath, and applyOperations are not mentioned in the docs. Are there other ways to get the same outcome? is there a plan to deprecate any of these functions?

[philcockfield]

Thanks @mshibl - that is very helpful for me to look at your process.

[nornagon]

You'll have trouble with that approach if two users edit a document
concurrently. The documents will get out of sync. You need something like
OT if you want to handle concurrent edits successfully.
On Sun, Oct 1, 2017 at 17:05 Phil Cockfield [e-mail]
wrote:

Thanks @mshibl https://github.com/mshibl - that is very helpful for me
to look at your process.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ianstormtaylor/slate/issues/259#issuecomment-333416493,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAKjAMR6CgIP6d01xYz43PrBqG9Zr5zZks5soCjHgaJpZM4JohSO
.

[mshibl]

@nornagon indeed after testing this turned out to be true .. now I see the point of having OT..

@philcockfield please be careful of the approach I've mentioned above as it does produce inconsistency between the two pages

[devilcoders]

@mshibl I tried almost similar approach about a week ago. As @nornagon already mention, it doesn't work right. I want to try https://github.com/automerge/automerge now. I will post here if I get any success.

[vshia]

@devilcoders @mshibl I've also looked into this and currently also looking into automerge. On that point, I'm running into a slight problem using Slate's Immutable data structure and automerge's built-in use of JSON. It should be solvable since they're all objects at the end of the day...
What do you think about setting up a channel on the Slate slack to discuss ideas on collaboration in general (whether it be OT/CRDT)?

[nornagon]

Automerge doesn't appear to handle merging edits within a string, so that will only work so long as your users don't want to edit the same paragraph at the same time :)

[aslakhellesoy]

@nornagon if a string is represented as an array of characters (strings of length 1), Automerge will merge them nicely. That's what I do in automerge-codemirror

[mshibl]

https://operational-transformation.github.io/ot-for-javascript.html seems like what we need .. I'll try to build something similar that works with Slate

[ianstormtaylor]

I've just added a #collaboration channel to Slack for anyone who wants to discuss this there.

[jeremysf]

Looks like automerge now has explicit support for text strings:

https://github.com/automerge/automerge#text-editing-support

[damo1884]

Any update on the status of a collaboration plugin?

[linonetwo]

I think we can try automerge now, some example here:

const Automerge = require('automerge');

// init
const doc1a = Automerge.init();
// another peer
const doc2a = Automerge.init();

// add something
const doc1b = Automerge.change(doc1a, 'Initialize with empty slate state', doc => {
  // from https://docs.slatejs.org/walkthroughs/saving-to-a-database
  doc.note = {
    document: {
      nodes: [],
    },
  };
});
// changes that should sent to peer
let changes = Automerge.getChanges(doc1a, doc1b);
const doc2b = Automerge.applyChanges(doc2a, changes);

// add some content to slate and serialize to JSON and add to automerge
const doc2c = Automerge.change(doc2b, 'Adding some text', doc => {
  // from https://docs.slatejs.org/walkthroughs/saving-to-a-database
  doc.note = {
    document: {
      nodes: [
        {
          object: 'block',
          type: 'paragraph',
          nodes: [
            {
              object: 'text',
              leaves: [
                {
                  text: 'A line of text in a paragraph.',
                },
              ],
            },
          ],
        },
      ],
    },
  };
});
// sent change to doc1's client
changes = Automerge.getChanges(doc2b, doc2c);
const doc1c = Automerge.applyChanges(doc1b, changes);

// change some text deep inside document
const doc1d = Automerge.change(doc1c, 'Rewrite line', doc => {
  doc.note = {
    document: {
      nodes: [
        {
          object: 'block',
          type: 'paragraph',
          nodes: [
            {
              object: 'text',
              leaves: [
                {
                  text: 'Rewrite this line.',
                },
              ],
            },
          ],
        },
      ],
    },
  };
});
changes = Automerge.getChanges(doc1c, doc1d);
const doc2d = Automerge.applyChanges(doc2c, changes);

// we can see the change is done correctly
JSON.stringify(doc2d);
JSON.stringify(doc1d);
console.log('`````');

// see the history
const historyPrinter = state =>
  `[${state.change.message}]: ${JSON.stringify(state.snapshot.note.document, null, '  ')}`;
const state1 = Automerge.getHistory(doc1d).map(historyPrinter);
const state2 = Automerge.getHistory(doc2d).map(historyPrinter);
console.log(
  `finalState1: ${state1}\n\nfinalState2: ${state2}\n\n finalState1 === nfinalState2: ${JSON.stringify(state1) ===
    JSON.stringify(state2)}`
);
console.log('`````');
// this can be saved to database at any time
const serializedContent = Automerge.save(doc1d);
console.log(serializedContent);

Parts of output:

changes = Automerge.getChanges(doc1c, doc1d);
[ { actor: 'fca6a267-8317-4471-a8ed-71267e7d5779',
    seq: 2,
    deps: { '46ede060-927c-4ddb-9224-a9c654c3d42c': 1 },
    message: 'Rewrite line',
    ops:
     [ [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object],
       [Object] ] } ]


// JSON.stringify(doc1d) is the same
JSON.stringify(doc2d);
'{"_objectId":"00000000-0000-0000-0000-000000000000","note":{"_objectId":"bc317181-4521-416b-b8bd-e4c5dfbd1b1f","document":{"_objectId":"6ba1520d-d1e9-4a89-9c09-6ca7be74792b","nodes":[{"_objectId":"2871dbd4-1a40-4d1d-9235-5505a9e76b0c","object":"block","type":"paragraph","nodes":[{"_objectId":"9ac26407-f32c-45cb-ad31-19726574a370","object":"text","leaves":[{"_objectId":"3e3d385c-2cb4-4e3d-b0c9-c0831d954656","text":"Rewrite this line."}]}]}]}}}'

finalState2: [Initialize with empty slate state]: {
  "nodes": [],
  "_objectId": "58274c9f-67af-4800-80c1-ce10f337a7fb"
},[Adding some text]: {
  "nodes": [
    {
      "object": "block",
      "type": "paragraph",
      "nodes": [
        {
          "object": "text",
          "leaves": [
            {
              "text": "A line of text in a paragraph.",
              "_objectId": "2c65869d-6d31-4d04-a6ed-c470a0f00775"
            }
          ],
          "_objectId": "1f0a10a3-287b-47ab-8041-72caa005f1d2"
        }
      ],
      "_objectId": "ce238a19-a36a-4e3e-adc6-63f81d6d1fcd"
    }
  ],
  "_objectId": "420b8b81-c506-4d61-959b-9ef53aab80b9"
},[Rewrite line]: {
  "nodes": [
    {
      "object": "block",
      "type": "paragraph",
      "nodes": [
        {
          "object": "text",
          "leaves": [
            {
              "text": "Rewrite this line.",
              "_objectId": "3e3d385c-2cb4-4e3d-b0c9-c0831d954656"
            }
          ],
          "_objectId": "9ac26407-f32c-45cb-ad31-19726574a370"
        }
      ],
      "_objectId": "2871dbd4-1a40-4d1d-9235-5505a9e76b0c"
    }
  ],
  "_objectId": "6ba1520d-d1e9-4a89-9c09-6ca7be74792b"
}

 finalState1 === nfinalState2: true

[jasonphillips]

I'm just going to drop into this discussion the recent work by Atom / Xray (the latter being a future replacement for Atom by its team, with the backend in Rust, and capable of running entirely in a browser via WASM). They have implemented a robust CRDT approach that is nicely optimized and thankfully allows strings at the lowest level, so it doesn't require each character to have its own key. Earlier in this conversation, it was said that CRDT doesn't have many complete examples in the wild, but it looks as if Xray is poised to change that in a very high-profile way.

An earlier overview is in this presentation: https://www.infoq.com/presentations/crdt-tachyon-collaborative-editing

The newer XRay repo is here, but you'll have to read Rust to get into the string buffer logic:
https://github.com/atom/xray

(That project also raises the option of some future slate-like library deciding to move all its data representations and operations logic into Rust as a wasm bundle running concurrently to the javascript code that handles DOM and events... the performance gains might prove substantial.)

[aslakhellesoy]

@jasonphillips is the CRDT you're referring to Teletype-CRDT or something else?

Teletype-CRDT works on a sequence of characters (strings), and cannot be used for rich text, which is represented as a tree.

[jasonphillips]

It's a variation / continuation of their ideas from Teletype, but significantly refactored now (and now written in Rust) as the core logic of their future editor. As-is, it doesn't fit a rich text scenario -- but their work on CRDT is still pretty useful it seems to me, particularly in how they reason about & optimize problems of finding correct fragments without costly offsets etc.

(the presentation linked is from many months back, but their recent work is in the xray repo)

[vshia]

This is still (very much) in the prototype phase. Nathan and I have been working on a bridge between Slate and Automerge. It converts Slate Operations to Automerge operations (and vice versa) to use Automerge's CRDT algorithm/library to handle merge conflicts and synchronize changes between clients.

Currently, the work only supports plain text and lists (using slate-edit-list). It doesn't support Marks yet (though attempting to begin something in this branch).

Let us know what you think. There is still _much_ to do, some of which is listed in the README, but wanted to get this out there for the community to look at. As a side note, there may be a few unforeseen bugs from the recent Slate update to 0.34... if you come across any, let me know.

[Arken94]

Hey all, and awesome work on Slate!

Now that slate has been made OT ready, I began to look at integrating the Slate editor with Firepad (Serverless firebase driven collaborative editing). Some of the contributors over at Firepad mentioned that it's simply a matter of creating an adapter for Firepad - so it shouldn't be _too_ difficult. It would be nice to allow people to embed a collaborative version of Slate through Firepad with just a couple lines of JS. The monaco editor was integrated into Firepad pretty recently - seemed like a fluid process.

I wanted to begin development pretty soon and was wondering if anyone is interested? Don't know much about OT and some extra hands would be awesome.

Thanks :)

[philcockfield]

Is the "syncing operations" sample the reference example for the work that came out of this issue? Trying to figure out what best to study to get to grips with this area.

https://www.slatejs.org/#/syncing-operations

Thanks

[majelbstoat]

@philcockfield: not exactly. it demonstrates synchronisation of low-level slate operations, but only in the same client. none of the more complex part of merging operations is handled in that example. in practice there would be a bunch of sophisticated merging code on the server to handle distributed collaborative editing.

[bryanph]

Instead of implementing operational transform, perhaps we can start with something simpler, like block-level locking, which locks the block for other users when the user requests a lock on the block.

[alexluong]

Is there any new updates on this issue? Have you made any attempt with FirePad, @Arken94? My company is looking at implementing real-time editor in the upcoming months, so I'd be happy to help with the research and experiment.

[meetbryce]

@alexluong I personally ended up opting for https://github.com/ProseMirror/prosemirror

[rahul1995]

@alexluong I personally ended up opting for https://github.com/ProseMirror/prosemirror

Isn't developing with Prosemirror a little tedious as compared to Slate?

[mitar]

Learning curve is steeper, but results are amazing.

[philcockfield]

@mitar - your comment "Learning curve is steeper, but results are amazing." refers to Prosemirror?

[mitar]

Yes.

[mitar]

I do not think this is a suitable place for such questions. Please turn to ProseMirror discussion forum and/or read through its documentation and examples.

[LionsAd]

I looked into SlateJS - Great work on the API. I really like it.

Here is some information that should be helpful to implement collaborative editing:

• Transactions / Transformations / Operations are only one part of the problem.
• You also need unique-IDs in the schema

While all the world is still implementing OT I would not suggest that approach.

OT's strength is text editing, but not rich text editing. There are numerous head-aches to make OT work with rich text editing and while ot-types/rich-text has come a long way, it still feels wrong (from an Architectural perspective).

CRDT on the other hand just makes sense [as implemented by yJS]:

You give every character within your distributed system a unique ID and you convert all transformations to operate not on ranges, but instead on those IDs.

Here is a blog post that can explain that better than I ever could:

https://conclave-team.github.io/conclave-site/

The summary is:

CRDT is modifying distributed state based on unique identifiers, while OT is transforming transactions.

Of course it is not ideal or performant to give every character it's own ID, but the way to solve that is to merge things together like yJS already does.

https://github.com/y-js/yjs/blob/master/README.v13.md#yjs-crdt-algorithm

So if one wanted to create a CRDT-ready data structure instead of using a pre-build CRDT type and syncing back and forth to it, then adding those IDs [they need to use a generator function provided e.g. by yJS] and a way to transform operations to work on those IDs is the way to go.

[steida]

Agree OT is no go. https://martin.kleppmann.com/2018/05/24/j-on-the-beach.html explains it nicely.

I checked yjs several times and I prefer https://github.com/automerge/automerge

This is a little bit obsolete, but the idea is awesome https://github.com/ept/slate-automerge/tree/master/src

[LionsAd]

Thanks - for our cases yJS worked performant and well.

What problems did you have with it?

BTW. Did you mean?

https://github.com/humandx/slate-automerge

The other link was not much to see.

[justinweiss]

Either OT or CRDT come with significant tradeoffs vs. the other, and whichever will work will depend a lot on your architecture -- I think Slate being agnostic about which to use is the right choice.

If you need peer-to-peer collaborative editing, you don't have much of a choice -- CRDT is probably the only way. On the other hand, if you have a central server, OT-based collaboration for Slate is straightforward (if time-consuming) to build. I haven't found a way to represent the intent of things like splitting nodes, merging nodes, and moving nodes using CRDTs. It might be possible to encode that behavior into the CRDT, but then they start to look a lot like OT :-). With OT, preserving that intent is easy, and easy to change. As interesting as CRDTs are, in my experience, rich text editing doesn't seem like a great fit -- though I'd love to be proven wrong. Plaintext with CRDT, on the other hand... ❤️

If you _were_ going to build CRDT-based collaborative editing in Slate, this paper is the most interesting approach I've seen.

[majelbstoat]

Just to say, OT is clearly not a no go, and is also clearly not unsuitable for rich text because OT is used for Google Docs collaboration. @justinweiss's point about nodes is also well-taken.

[LionsAd]

Longer response to the paper OT vs. CRDT in real applications here:

https://gist.github.com/LionsAd/19673619c2fa438e475ddca9aa2841b3

@majelbstoat Yes, but the question is: How complex does it need to be to achieve that? I got really
scared by the tales of the CKEditor 5 people.

@justinweiss The only tradeoff that CRDT has is the tombstones and the lookup tables needed to find the position of the IDs, but CKEditor 5 also needed a graveyard for their OT implementation. I also outlined some thoughts in my linked post that epoch based GC with a central server will work well for removing the tombstones and hence is not a problem in practice.

But now as a conclusion I think that CRDT vs. OT is a pure theoretical question and does not even matter in practice for our modern JSON-model based editors.

The thing is always the same (OT vs. yJS) as example:

• Convert JSON model to plain-text / YXmlFragment
• Apply OT / CRDT operations to converge the state
• Convert plain-text / YXmlFragment back to editor state using either diffs, complete loading, or object lookup + change transformation

I am not seeing the overhead of either OT nor Y here being any different.

And in fact even implementation wise - let's assume we have rich attribute changing operations it is the same conceptually:

• $table->setRows(3)

is converted to:

• Within the OT-plaintext / Y-Model:

• Find the (serialized or deep nested) table object

• Set the attribute 'rows' to '3'
• Sync that change
• Convert model back

But we can do better!

But it can be kept with both CRDT / OT also as a rich operation:

• setAttribute('table', $position_or_id_of_table, 'rows', 3);

and again within OT the start_position of the table would be changed in the operation, while in CRDT it would just reference the ID of the table.

If the table was deleted then the OT operation would be removed as invalid, while the CRDT operation would be applied to the tombstone (or rejected as well as the object is invalid).

Here is a little bit of a difference of OT and CRDT in case of undo, which should preserve the state done while the table was not there anymore:

With CRDT if the table is restored by an undo operation it would have the change made on the tombstone [if that was chosen], but with OT the server would need to keep track of invalid operations and apply them when the undo operation makes that operation valid again (feels harder, but doable).

So the simplest to optimize editors is to allow frameworks like either Y or OT to be able to directly find the objects on which to execute the state change (and it does not matter if that is position or ID based).

One possibility could be a hash table of unique_ids to the immutable objects. Another could be to keep track of all generated objects within an array and having each object know it's ID.

The lookup via plaintext-OT is then as simple as to find the ID within the object and if that is always the first property the serialised structure will always start with e.g.:

{'id': 1234...

and then the object can be loaded and the state change applied to it.

Note: These are just optimizations to avoid the whole diff/reload approach and _keep_ rich object operations where ever possible. And that overall matches more how react works and the original intent of slate.

---

I am not understanding why node splitting is a problem if you could convert the editor JSON to a plaintext model as well and apply plaintext-CRDT then, where it seems to be simple.

Because as we have a 1-1 isomorphism between both models if it is simple in one model, there should be a function that makes it possible in the other model as well.

I still think rich operations should be what is send and applied and if we mix OT and CRDT - who cares as long as its simple ;).

[LionsAd]

Okay, node splitting again:

So the famous example:

• User A makes a paragraph node into a list
• User B splits the paragraph in two paragraph nodes

I am not seeing why this is a problem with CRDT:

The paragraph consists of a sentence of e.g. 42 characters:

Lets assume the client ID is 4 and the counter is 100 and all was inserted by this client:

The range then to apply the list operation is then: 4-100 to 4-142

makeList('4-100', '4-142')

If the paragraph is split by client 5 at e.g. 20 the IDs of the characters (including the new line / paragraph break) are:

4-100...4-120 5-0 4-121...4-142

The makeList operation will still be applied to all characters and hence to both paragraphs.

If it comes first, the whole 4-100...4-142 is made into a list and if it is then split at 4-120 the 5-0 is still inserted correctly.

What am I missing? As long as every character + rich object has it's own ID ranges can always be found.

Moving is a problem yes as it's delete + insert in CRDT and leaves a lot of tombstone records, but it's not trivial in OT either as far as I've understood.

[LionsAd]

I finally understood what the real problem of OT / CRDT is [what the CKEditor guys have talked about] and why those simple cases pose such a problem in Editors:

• Editors use really low level operations, so instead of makeList() / removeList() everything is text based.

e.g. in slate making a list is the following operations:

0: {object: "operation", type: "set_node", path: Array(1), properties: {…}, newProperties: {…}, …}
1: {object: "operation", type: "insert_node", path: Array(1), node: {…}, data: {…}}
2: {object: "operation", type: "move_node", path: Array(1), newPath: Array(2), data: {…}}
3: {object: "operation", type: "move_node", path: Array(1), newPath: Array(2), data: {…}}

This then leads naturally with automerge / CRDT, but probably also naive OT implementations to conflicts, because they only operate on text or JSON, but not on semantic structures.

The case for high level state changing operations

This explains a lot of confusion of me, because I see the text editor universe as a set of high level commutative, inversable and potentially idempotent operations that operate on objects to change state (which is similar to what the OP proposed using redux operations).

In addition they need to change text, but that is a solved problem.

Hence I have two things:

• Internal state (not synchronized, but used for the editor model to display)
• External state which is a flat projection of the DOM model to apply operations to (synchronized via high level operations)
• High level (CRDT) Operations that transform this internal state (synchronized across the wire)

The advantage of a high level operation (as long as we can prove that it is commutative and inversable) is that it should be more user intent preserving:

• If I make a list out of a selection (implicit or explicit) I have two "identifiers" that are showing the range of the list operation

• If someone changes something within this range, I still apply the list operation to this range

• If I undo it and someone has continued to write more list items, this will undo the whole list operation, not just the case where I did that
• If I redo it, then again it would continue to be applied on my original range (it's the best we can do)

In many cases even the more naive prosemirror reconciliation collab approach would be able to transform a selection to be user intent preserving on this high level operation.

Just when receiving such a high level operation would the editor change the internal state in a way that it makes the list, similar to how the DOM is rendered independent of the internal editor model.

Open questions:

• What happens when two users make a list, one undo-es it? [as the operation was on the non-list state originally, it's likely it would just un-list it as both made the same change (idempotent)]

• What happens if there is an object in between, which splits the list as it cannot be in a list item per schema? (The auto-schema would solve that by splitting the list automatically again once undo / redo of makeList)

• When happens if a user intentionally split the list, now someone undo's and redo's it? [even if the list would no longer be split but be just a list-item again, that would be 'acceptable' in most circumstances -- Google Docs handles it worse as undo does nothing in the conflict case.]

Thinking of that, maybe it would be possible to model this as:

{list}Item 1 Text...{no-list}...some more...{no-list-end}Item 3{list-end}

which would be intent preserving, but then the problem is that just {list}{list} would be overriding {no-list}, etc. which makes things really complicated.

Splitting the undo-make-list operation obviously could work on the new identifiers (which is more OT obviously), but let's assume a client that does never read new state, but still pushes out his state changes (possible in CRDT p2p).

And all it does is: undo / redo / undo / redo the makeList operation [or just send makeList / removeList / makeList / removeList / ... alternating on the same range of identifiers].

A possibility to design a split of a range would be to have a splitList operation.

In other words:

A given range in the context of 'list' is mapped to two ranges instead.

Maybe ...

I can now see the merits of quill's delta format.

Okay, let's assume we model the splitting of the list as just removeList(), then it's not longer commutative, because:

• makeList(0,10)
• removeList(4,5)

and

• removeList(4,5)
• makeList(0,10)

do not have the same result, which means currently the makeList / removeList are not commutative on ranges, the same is true for 'marks'.

The question then is: Is it possible to derive a set of high level operations that are user intent preserving AND have the CRDT properties?

TBC ...

---

Okay, I understood an assumption I made:

• The removeList is only a meaningful operation if there _is_ a list to remove (e.g. as a user I must observe a list to be able to make a removeList operation)

• Lets assume there is a list from:

3 - 6

One user removes the list from 4-5 and leaves just positions as lists (3,6), another user increases the list from 0 - 10 (he has observed the list from 3-6).

User A does: removeList(4,5)
User B does: makeList(0,10), which is transformed to: makeList(0,2) + makeList(7,10)

Regardless in which order the operations come in, they are non-conflicting.

However when User B had not yet seen the state, then this does not work -- so the OT approach of having the server dictate to first add all remote operations is needed here as well, which I think is okay as it has several other useful properties.

Still the intent of the user might have been to just do a list from 0-10 - regardless if someone else had removed the list or not.

So while the result is nicer it's still in conflict.

And the same problems are true for marks as well: Adding is easy, removing is hard.

However the same is true for CRDT in general: Adding is easy, removing is hard and a tombstone approach is used for removing, so maybe here a tombstone approach needs to be used as well?

TBC ...

---

So how does Google do it?

In Google marks are at least in my testing independent structures that operate on ranges in the document [maybe on a per paragraph basis?] and that are transformed to extend or shrink when the range changes. [at least in the conflict case of mark-bold + split node Google Docs correctly made the whole thing bold -- no idea what happens on removal of mark-bold within the middle]

However lists / list items are different from that and seem in Google Docs to be separate nodes:

Google applies the 'makeList' operation to one node, but not the other.

This implies to me that when Google is splitting a node, that one is kept and the text moved to another node.

---

Researched this some more and internally it seems the operations are converted that lists have both a toggle list thing, but also a depth.

Undo-ing the operation to make the list from 0-10 then re-doing it leads to the funny state of a list that is present for all 4 items, but has a depth of 0 (no-list).

[LionsAd]

I thought some more about this and also researched Google Docs some more:

• A very simple (inefficient, but should be working) model would be to have each character be the main object of the document (therefore it is way more like text-CRDT / text-OT)

Instead of having:

{'some text', marks: ['bold']}, {' that continues here'}

You just have:

'some text that continues here'

And then you have an object / position map, that maps 's' => {'bold': true}, 'o' => {'bold': true}, ...

A paragraph also is just an 'enter' like in normal text with contextual information.

There are also 2 different styles in Google Docs:

• Text styles [applies to characters]
• Paragraph styles [apple to paragraph objects]

And then there is a table, which can contain other text objects, which again have text styles and paragraph styles.

By removing all the nice things that make the editor model so nice to work with (rich-deep, dom-like things) in the model used for CRDT / OT synchronization, the implementation gets simpler.

Every character has it's text styles.

Every paragraph [ends with enter] has it's paragraph styles and indent is just a counting property, list_style is just a enum property, left / right / blockquote is just a style property on the paragraph.

This brings you 90% to Google Docs.

The advantage of character based styles and whole text operations is that the whole conflicts that lead to incomplete node_split or conflicts with duplicated text are avoided.

Last change wins - for properties - also is closer to user intention usually.

It is also much closer to normal OT / CRDT.

For rich text things like tables there would still be conflict, but that is okay -- because the HUGE problem with lists + paragraphs + markers is that they can be converted into each other and lead to node splits, etc.

By not having that problem in the first place, it is mostly avoided.

[sepehr500]

So just to clear the air, is Slate js ready for collaborative editing right now, and if yes, has anybody actually done it in production?

[tommoor]

Yes. At least two folks have separately achieved it in a production environment according to conversations in the Slack group, unfortunately neither has open sourced their work.

[justinweiss]

Yes, at Aha!, we have built a collaborative editor based on Slate. We use Operational Transformation on the Slate operations, which meant writing around 100 transformation functions -- luckily, most were pretty straightforward and could share a lot of the same code.

Once you have that layer, you can also use OT for collaborative cursors and collaborative undo / redo, which is nice. As usual, the details are hard to get right, but it's definitely possible in Slate. Permissions, snapshots, integration with other systems, etc. make things much more complicated :-)

I wrote an article about creating a collaborative editor in general, which goes into most of the basics you'd need to know. It's based on a talk I gave at RailsConf, so the examples are in Ruby, but the ideas translate to whatever language. The CKEditor collaboration article is also useful for learning about some of the other things you'll need to get right in order to make collaboration work. Their "post-fixers," for example, are probably similar to Slate's normalizers.

[t3db0t]

@justinweiss So....—any chance that y'all are open-sourcing your collaborative editor? Eh? Eh? ;D

[TheSpyder]

There are a couple of partial solutions around for collaboration in Slate, although both seem to have reached a certain point and then stopped. It's quite a lot of work to implement all the necessary transforms.

Mentioned above is CRDT with AutoMerge:
https://github.com/humandx/slate-automerge

There's also OT with ShareDB:
https://github.com/qqwee/slate-ottype

[t3db0t]

@TheSpyder Awesome, thank you :)

[cudr]

Created plugin for co-editing: https://github.com/cudr/slate-collaborative

Hope, this will help for someone

[gnestor]

I think I speak on behalf of most of us when I say thanks so much for sharing @cudr!! 👏👏👏

It's great to see an open-source plugin implementation. I tried it out and it looks great. Can you share a bit about your take on it: what works, what doesn't work, future work, etc?

[cudr]

@gnestor, thank you!) I test it couple of times, and it works correct together with popular plugins. But i haven't time a lot to check this out at all. It will be great if community helps with this, and give feedback about how it works in real (prod).

Welcome to contribute!

[ianstormtaylor]

This stuff is all now possible with Slate. I'm going to close this issue since there's not much left for core to do, but feel free to keep discussing!

[Tamriel]

@cudr: You combined Slate and Automerge - great!
@ianstormtaylor: Can you elaborate a bit? How are conflicts handled?

[TheSpyder]

@Tamriel Slate doesn’t do collab out of the box, it has the api available to implement it.

[amilich]

@cudr any plans to update for 0.5+?

[Immortalin]

https://github.com/Immortalin/slate-operational-transform

[Immortalin]

^for Slate 0.5+

[t3db0t]

Really cool @Immortalin ! Do you expect this repo to receive updates and improvements, or stay as a demo?

[Immortalin]

It will most likely stay updated as long as I continue to use Slate in production. But it is primarily an example/reference. However I would love for Slate to have first party integration with ShareDB like Quill.

[Immortalin]

The operations that are currently missing for the ShareDB JSON1 type is merge node and split node. If Slate can offer first party support, it will make a lot of things easier. My solution of using json0-ot-diff is more of a hack than anything. Running a search algorithm every keystroke is not optimal. Don't get me wrong, it works fine and the lag is not too significant even when pasting large documents. But a lot of cycles are wasted since Slate already has Operation types and we are not reusing them in my example.

[Immortalin]

@ianstormtaylor https://github.com/ottypes/json1/blob/master/README.md

[TheSpyder]

This issue was closed months ago, as far as I know Ian isn't interested in offering first party support for RTC. Just the framework to make it possible.

[cudr]

@cudr any plans to update for 0.5+?

@amilich Already updated

[BitPhinix]

I created a yjs based collaboration plugin with good performance on large documents: https://github.com/BitPhinix/slate-yjs

(Tested on the newest slate version)