Codesandbox-client: Add a collaborative mode

[Telokis]

I think adding a collab feature similar to jsfiddle's one would be a nice thing.

I am currently trying to find a good sandbox to teach my friend how to use React and I would really love being able to do that on codesandbox.io!

[CompuIves]

Ah, interesting. Realtime updates is a good feature to add, very useful in classroom scenarios as well.

Are both users able to edit in the collab mode of jsfiddle?

[Telokis]

Yes!

Both people are able to edit simultaneously. I can ever see my friend's mouse pointer moving in real time and there is a little bubble when he clicks somewhere.

You can see it quickly by looking at this link: https://jsfiddle.net/Telokis/syco2sfu/#&togetherjs=OOfThS7L0k

[CompuIves]

Our transcript in the fiddle for 'documentation purposes' 😅

tivar Hello = React.createClass({
  render: function() {
    return <div>Hello {this.props.name}</div>;
  }
});

Wooow this is cool
Yep!
ReactDOM.render(
  <Hello name="World" />,
  document.getElementById('container')
);
Except those weird cursor jumps!
Also good chat application ;)
Yap, noticed them

But this will be pretty interesting to implement, I think we could take something from the google docs paper


https://neil.fraser.name/writing/sync/

Let's do this step by step, first the websocket functionality to let the master only edit. Then start working on letting both edit.

Sure, no need to rush things out! :D

This feature will be fun!

Damn that doesn't compile! :(

Heh, strange :')

[kanoshin]

Could be super-super useful to run interviews with remote developers. Currently we are using Hackerrank Codepair for it, but it doesn't support React.

[alsuren]

I would be interested in helping out with this. I'm going to have a look around over the weekend to see whether there are any other open source editors that do collaborative editing already, but I like how codesandbox can do offline compilation (with a bit of webrtc magic, you might even be able to support local collaboration in countries with poor internet). I will also ask around at the mozilla festival, and see if anyone else is interested in helping.

edit: oooh! this looks interesting: https://github.com/y-js/yjs

Collaborate on text. Supports two way binding to the Ace Editor, CodeMirror, Monaco, textareas, input elements, and HTML elements (e.g. <h1>, or <p>)

[alsuren]

After a bit of research, I found that jsfiddle and kodeweave and a bunch of others are now using https://togetherjs.com/ . I expect this may be the way to go.

[CompuIves]

Wow, what @alsuren sent is really interesting. I've investigated both and they both seem solid choices, togetherjs looks more plug 'n play though.

I tried togetherjs by just adding the script tag to the bundle, and it worked! I had this overlay and could see the cursor of others. I noticed some issues though. Togetherjs propagates clicks and keyboard events, but this doesn't work if one of the users is in a different file. We could maybe send these messages manually. It also didn't recognize changes to the code, but that could be fixed as well.

I then looked at yjs, this looks like it's more fine grained, which is very nice. We have the same challenge with multiple files though.

It's a coincidence, I went with @christianalfoni to Reactiveconf the past few days and we've talked a lot about this functionality. He's the creator of webpackbin and built the live feature of it as well. We were thinking of building it together for CodeSandbox, we could use all the help we can get!

So, our idea was (before we saw yjs and togetherjs) to first change how we see changes in the code editor. Currently we set the new code state by just updating the whole code, but we should move it over to a sequence of actions (eg. insert 'hello' on line 5, column 2). Then we could give the owner of a sandbox a button to share their session. They will get a new url then and others can see what the owner does by having a websocket connection with our API server. The API server broadcasts all API results (eg. 'create new module') and will broadcast all code changes the owner makes.

Now this was our idea before I saw y-js/yjs and togetherjs.com. Maybe we can find some middle ground. Would be great to have some discussion on this!

[christianalfoni]

TogetherJS does seems pretty amazing and a quick prototype would be interesting. But there is an other mode than "collabarative" that would be very good for Codesandbox and that would be "teaching". When we manually control the shared state of the application it is possible to give a "teacher <-> students" environment, where the teacher controls who can control the sandbox, and lock any interaction from students out when not in control. This is how Webpackbin works today. There is also the fact that togetherjs is completely generic and might prevent us from adding custom functionality. It is DOM manipulation based, whereas the Webpackbin way is based on the actual state of the application. This will affect things like where the iframe needs to refresh, changing files etc... these are not really DOM manipulations, which might cause problems with togetherjs.

Absolutely not saying togetherjs is not a viable option, just giving input to the conversation :)

[alsuren]

Some thoughts after talking to one of the thimble engineers ( @gideonthomas I think ) at mozilla festival:

Their tracking label for the collaboration issue is https://github.com/mozilla/thimble.mozilla.org/issues?q=is%3Aopen+is%3Aissue+label%3ACollaboration+sort%3Aupdated-desc

To summarize what he said (this is now 3rd hand so I might have something wrong):

• They've got a GSoC student working on it recently
• They rejected TogetherJS because it all feels a bit hacky to work with.
• An initial prototype was done with yjs but their impression was that it did quite a lot of things that they didn't need (google-docs style cursor synchronisation), and it didn't integrate with their text editor of choice (brackets).
• They're currently working with another library (https://github.com/Operational-Transformation/ot.js/ I think) and trying to integrate it with brackets directly (He said that thimble.mozilla.org can be thought of as basically a persistence layer for brackets, so this works for them).
• I feel like if they're going to end up with a re-usable component, it's likely that it will be a synchronisation layer for brackets. This may or may not be directly useful for us.
• He encouraged me to make some noise on their tickets, so I'm going to link to the most interesting ticket that (the prototype brackets integration that was mentioned. Hopefully this should create a mention pingback so they know that we're trying to solve a simliar problems :D ) https://github.com/mozilla/thimble.mozilla.org/issues/2407

[alsuren]

After looking around for far too long, it seems like there are a number of different approaches. Operational Transforms and CRDTs are the main players.

(Side-note about read-only monaco panes: monaco has a readOnly bool that you can pass in, but doesn't have a way to say "yeah, but let any edits from the synchronisation library" as far as I can tell. I suspect that it would be possible to implement, but it would need to make it into the main vscode repo if you wanted to avoid maintaining a fork for the rest of your life.)

Most of the OT-based libraries feel unmaintained (last serious commits 1-2 years ago) and the CRDT-based ones feel like they are still in the "maintainer is excited about this new hotness" phase but might die out again later. They mostly come in a core-repo-plus-plugin-repos model, so it's difficult to work out which has the best community, but here's a link that attempts to compare them anyway:

https://npmcompare.com/compare/automerge,gulf,ot,scuttlebutt,yjs

Here's a few highlights as I think of them:

• Automerge is one that I have only just stumbled upon. It is CRDT-based and looks fairly new, but already has a handful of contributors to the main repo. It feels like it's got some industry+academic collaboration behind it, so it might actually have legs. It has a codemirror integration by what looks like a 3rd party and a name that sounds pragmatic, rather than just an academic project.
• Gulf is OT-based and declares itself to be unmaintained. Its maintainer has also written plugins for scuttlebutt and yjs, by the look of it. There is a project (hivejs) that says it does cursor synchronisation based on Gulf, but its domain name has lapsed so I didn't bother to check it out.
• ot.js looks like a "let's implement this algorithm in many languages" project, but it doesn't feel polished, and feels unmaintained.
• scuttlebutt (I've still not read the paper for that so I'm not sure which category it falls into) is something that I vaguely heard of back in the day. Its maintainer seems more interested in something called secure-scuttlebutt, but still maintains the old repo. It has a codemirror integration and a concept of a read-only stream.
• yjs is the first one I came across. From reading the sources, it feels like the core of it is a CRDT, but the author started it as a bachelor's project and doesn't point to any of the background literature that inspired him. He is still paid to develop it by the same university by the looks of it, and it has a bunch of plugins (including an IPFS plugin) contributed by a third parties, but it is literally impossible to find them all because none of them declare peer-dependencies on y-js. The text plugin can bind itself to codemirror and monaco out of the box.

If I was forced to start this project tomorrow, I would probably have a go with y-js and see whether read-only streams can be managed via a transport plugin and cursor synchronisation can be managed via an extension of the y-text type plugin at a later date. I would also avoid their webrtc integration and just use the websockets one.

If there is no rush, I think I would wait for Automerge to either become stable or die out. There seems to be a lot of hype around offline-first and CRDTs at the moment, so it may be that a better library emerges as time goes on.

The other thing that might make my choices different from yours is whether you are expecting to do all of your building on the client side or the server side. I don't have access to your backend, so I don't know how easy it would be to plug in a CRDT/OT library to decode the current state of the file and build it. I have assumed that you will be doing all of your compilation client-side going forwards, so syncing the files will be enough.

I actually don't have the head space to work on this at the moment, so I think that's all I'm going to contribute on this issue.

[CompuIves]

Wow, @alsuren this is reaaally great!!! Thanks for such a clear summary and verdict. This really helps.

From your summary I think that we should go with AutoMerge in the future, and build a version now that lets 1 person edit the sandbox while others watch. I think that that already covers 70% of the use cases and in that case we can work on a simple implementation, then later move on to Automerge for a more sophisticated solution.

I've done some talking with @christianalfoni and we think that it would be best to approach is this way:

Show a button for the owner of the sandbox to create a live session, the owner gets a url that they can give to users to join the sandbox. The users see all operations of the owner on the sandbox (edit code, add dependency etc). The owner can 'transfer' control to a user and take it back on a moment.

It would be great to utilize websockets for this. So the owner sends all code changes to the backend and the backend broadcasts these operations. The backend also broadcasts all API results to the sandbox (like renaming a file). This is a great way to utilize the concurrency of the Elixir backend.

When we have this working we can re-evaluate Automerge and just create an extra mode that removes the owner check. We will then use Automerge for all code operations.

I'm planning on finishing the server part tomorrow. What does everyone think?

[christianalfoni]

@CompuIves There is another alternative for collaboration. When you first have the websocket communication that is pretty straight forward you can add a synchronization layer to it. Basically this is what happens:

1. An operation is performed at a client. A message is past to server with identity of user running it
2. Server timestamps the operation and sends it to everyone, including the original client
3. The clients can now keep control of the order of events, and what are their own
4. If a client detects wrong order of events it can reset state to last snapshot and replay the operations again. It can also ignore events owned by the client if ordering is correct, not replaying the operation

I am not sure how Codemirror/Monaco curors act when doing these operations, but hey... you need some challenges, right? :)

My point here is that it gives complete freedom in what to collaborate on and with what technology. It is all state driven :)

[CompuIves]

@christianalfoni I agree, that approach sounds really solid.

[alsuren]

@christianalfoni in step 4 rather than reverting your own action when an out of order one comes in, the idea behind OT is to transform the out of order event into one that is equivalent to reverting and reapplying your own event. Whichever way around you do it, you need to transform either your event or the event that just came in, and avoiding the revert/reapply of your local event makes everything less sad.

If you decide to write this from scratch then you will at least want to use gulf or ot.js (or the OT bit of togetherjs) as a crib. Having the central server doing the timestamps might be enough to let clients work out the current state of the buffer (I think), but if the server wants to know the current state too (for server side compilation etc?) then it will need to transform incoming operations too.

[christianalfoni]

@alsuren Thanks for the input here! I am not familiar with the transforming of the event, sounds interesting! Though at the same time that seems very complicated? If we use DOM updates as analogy I think of OT as handling the specific event and update the respective DOM content... the approach I listed is more like templating. You just render everything again with updates state. Simpler, though not as performant of course... but rebuilding state from snapshot would be crazy fast anyways.

THAT SAID! :D I have not implemented this stuff before, so thank you so much for the input. There is probably a very good reason why OT exists :)

About server knowing the state it changes stuff, definitely. Though I can not see any reason why the server would want to know the state other than optimization... but it will complicate a lot. For example instead of the client asking the host for a snapshot of the state, it could just get it directly from the server. It is theoretically faster, but in reality it does not matter... and you avoid a complicated implementation on the server... IMO :)

[alsuren]

What I mean by transforming the event is described here: http://operational-transformation.github.io/what-is-ot.html (in their example, they have Bob sending insert(0, 'B') and Alice sending insert(12, 'A') concurrently, so Bob has to turn Alice's event into insert(13, 'A') before applying it to his DOM in order for the 'A' to appear in the right place. There are a bunch of other cases that other people have have worked out how to deal with already (and even discussions about how to deal with some fun edge cases like overlapping insertions and deletions), so I would probably just pick a library and use it/crib from it, rather than going down the rabbit-hole of finding all of the edge cases yourself.

[xyc]

FYI supports of collab mode come to some major code editors:
Teletype for Atom: https://news.ycombinator.com/item?id=15704730
Visual Studio Live Share: https://news.ycombinator.com/item?id=15704376

[CompuIves]

FYI supports of collab mode come to some major code editors:
Teletype for Atom: news.ycombinator.com/item?id=15704376
Visual Studio Live Share: news.ycombinator.com/item?id=15704376

Wow, that looks really cool!

[trevorwhealy]

@CompuIves hey Ives, what's the word on this? Doing real-time collab front end interviews in codesandbox would be a dream - I've tried watching interviewees screens via Skype, Hangouts, etc and it was mostly great, but I really missed this feature.

I'd like to introduce this to more people in our org, but many will probably require real-time collaboration (at least viewing in real time, if not editing).

[CompuIves]

Hey @trevorwhealy. There is some progress on this, the redesign allows us now to start building the realtime collab mode. I'm excited for this feature, so I'll start working on it today!

[RohovDmytro]

@CompuIves superb news! Can I dare asking about ETA? I am thinking how to organize my class with a student and collaboration mode is a perfect fit for this.

[empz]

+1 I really want collab mode in Codesandbox. I would certainly pay for such feature!

[CompuIves]

Hah I've been working on this for the past few weeks and it was soooo hard not to respond on this issue! Well, it's here now 😄

https://medium.com/@compuives/introducing-codesandbox-live-real-time-code-collaboration-in-the-browser-6d508cfc70c9

[bora89]

Very cool feature, but I have found that once I enable Live - VIM mode disabled immediately. Is that by design so?