Slate: Proposal for Sponsored Open Source Android Support

[thesunny]

I am considering a sponsored version of an Open Source Android Plugin for Slate for USD $25,000.

1. This is an alternative to the proposal for a Paid Android Plugin (also proposed by me) for USD $500 per license as described here https://github.com/ianstormtaylor/slate/issues/3573

2. The Android Plugin will be released as open source under the same license as Slate and will support at least the latest version of Android with similar features and limitations as the 0.47x Android Support

3. Building Android support is a lot of work. We would love to be able to contribute for free (like Ian!), but the cost is high to burden alone. We contributed Android Support for Slate 0.47 which was working although not perfect and that cost about USD $40,000. That doesn’t include research we did prior to starting or the devices we had to purchase to test on.

4. It's uncertain how much it will cost to complete the 0.50x version of Slate Android. It may cost more or it may cost less. Either way, we are committing to completing it if we raise the funds. If it costs less, we will not be returning unused funds; if it costs more, we will cover the difference. Note that the 0.50x port is not trivial as another user worked for 3 months on it and it was never completed.

5. For clarity, we already spent USD $40,000, so even if the port costs $0, there will be no profit from this.

The port will be completed in partnership with Wayne Leroux with assistance from me. Wayne has built an impressive collaborative editor based on Slate (https://github.com/wleroux/slate-collaborative) so I have confidence in his ability to complete this project. He has also contributed other open source projects (https://github.com/wleroux?tab=repositories). I expect him to be able to complete the project; however, if for any reason he cannot, I will either complete it myself or will return the funds. Wayne and I intend to work together on a different Slate related project which would benefit from Android support.

If we are unable to raise the funds through sponsorship, we will revert to the paid model at USD $500 per license. For clarity, if we don't raise the funds through sponsorship and use a paid model but later receive the equivalent revenue in licensing fees from the paid model (USD $25,000 or 50 licenses), we will not then open source the code.

I recognize that this is a non-trivial amount to raise. I expect that companies that can afford to do so, will need to contribute more in order to meet this fundraising goal. To put the amount into perspective, this is the equivalent of contracting a senior developer for about 6 weeks. 3 organizations, contributing the equivalent of a contract valued at 2 weeks worth of work will reach the sponsorship goal (about USD $8000 each) or 50 smaller contributors paying what would be the licensing fee of USD $500.

Update: Now Live!

Add Android Support to Slate - KickStarter Campaign Now Live

[thesunny]

Hi everyone, just a quick update:

I've put together a KickStarter campaign and have submitted it. We are awaiting its approval. The campaign is in Canadian Dollars (we are a Canadian company) but we are raising the Canadian equivalent of the US dollars noted above.

We've started work on the Android Plugin part-time at the moment.

If we are able to raise the funds, our plan is to start work on the plugin full-time in August and upon completion release the plugin as an open source project.

As a reminder, if we are unable to raise, we plan to release the plugin under a paid license.

[signalwerk]

@thesunny I hope you also have thought of small tiers in the kickstarter campaign. I don't need Slate in a commercial product or client but would also like to contribute, since I have some private projects with slate. Plus I like what you did for 0.47!

[thesunny]

@signalwerk I have a pledge level at $50. Do you think that works?

[signalwerk]

@thesunny awesome! 🥳 can't wait to see that happening!

[thesunny]

Hi Everyone,

The KickStarter campaign is now live.

We will be grateful to all contributions towards the goal no matter how small.

That being said, in order to make this campaign a success, in my opinion, we will need both of the following:

1. Companies that use Slate for their business to contribute at a corporate level. It will be difficult to meet the pledge goal without this support.

2. Members of the community (people like you) to promote the campaign to your organization or to other organizations that would benefit from Android support in Slate.

Many have asked for this fundraising towards open source Android support. The work has been done to create the fundraising campaign but we need your help in order to make this a reality.

Support this fundraising campaign:

Add Android Support to Slate KickStarter Campaign Now Live

Note: If we raise above the fundraising goal, we will spend time to support earlier versions of Android.

[thesunny]

An update,

The KickStarter campaign

We have raised $2,351 out of $30,000 with 10 contributors. This means that 40% of the people that showed support for the Android plugin, have backed the campaign.

If you intend to back the campaign, please do so now and help build momentum.

Several of you asked me to seek sponsorship in GitHub issues, on the Slack Mobile channel and privately. To make this happen, it requires help from everybody to get it to the right people. For clarity, this is not a profit making venture for me or my company (we will lose money), but it is a way for the community to get a much needed feature into Slate.

[clauderic]

I've created a 12 page design document which includes all of the issues I encountered building Android support the first time along with the solutions I used to overcome those issues.

Hey @thesunny, the Kickstarter campaign mentions a 12 page design document. Is this something that can be reviewed by the community?

The Kickstarter campaign is scarce on technical details on what the proposed approach would be, and I think it would help build confidence in the project if more technical details were shared.

The previous solution revolved around composition start / end events and MutationObserver.
Is this still the planned solution moving forward?

I've spent the better part of the last month reviewing the Android plugin that was shipped in [email protected], and am really appreciative of the work that you put into it. It has a number of issues that I've since been trying to fix.

While trying to fix bugs in the original Android plugin, I noticed that there are a number of things that I think were overlooked with the original solution:

• Composition events do not fire consistently the way you might expect them to in Android. For example, the compositionend event often only fires when the keyboard is dismissed, or sometimes fires out of sequence at the beginning of a composition, and therefore should not be relied on to determine the end of a composition
• The Slate value needs to be updated on every single text insertion / deletion / line break to allow for other plugins to interact on the value. The previous Android plugin only attempted to reconcile the mutations that had occurred on compositionend, which only happens when the keyboard is dismissed or the input is blurred as outlined above.

• Many plugins rely on keydown events to override the default browser behaviour (for example, on backspace or on enter). On Android, keydown events have unidentified keycodes, and therefore cannot be used reliably by plugins to augment the default browser behavior.

  In an ideal world, plugin authors should not have to write plugins with a separate code path for Android, Slate should expose a universal way for listening for specific events such as backspace and enter

• Plugins can also affect the DOM and trigger changes that are then detected by the MutationObserver. There needs to be a strategy in place to distinguish between mutations that originated from a composition and mutations that originated from plugins. As outlined above, the compositionstart event is unreliable on Android, in my experience, the beforeinput and input events are a more reliable way to detect the start of a composition
• Some actions are extremely difficult to reconcile using MutationObserver alone. The main example that comes to mind is handling backspace properly, which can result in dozens of mutations that are difficult to decrypt. In my opinion, some actions such as backspace would be better handled using beforeinput rather than MutationObserver alone
• The original solution uses the ReactDOM.unstable_flushControlled method, which is no longer exported from the stable builds for ReactDOM as of react-dom^16.11.0

I am sharing these issues not to criticize any of the work that was done, but rather in the hopes that we can discuss them along with potential solutions so that these aren't overlooked in the next Android plugin.

[thesunny]

Here is the Design Document although I suppose it's more of a "How I got 0.47x Android to work".

https://airshelf.com/thesunny/slate-android

I read all of your comments but I don't have the time to address them individually. They all make sense and I think I considered most or all of them at one point. One of the problems is Android is just not simple. I tried many more direct approaches and spent, I think, 2-3 months going in circles on the "without mutations" version for example. Some of this is documented in the link.

Also, keep in mind that if you are using it with the latest version of Android, it was never tested on it because it didn't exist. If it works at all, it's kind of lucky. The new version will be tested on the latest version of Android.

One thing that may never work is being able to intercept certain keypresses. The reason why is that Android may not provide that feature (haven't checked latest version of Android). You can't preventDefault some events that can be done in every other browser and that is by design by the Chrome Android team. This is also the reason why I have to revert the DOM a lot.

[thesunny]

Hello everyone!

Some great news is that we have one Silver Corporate Sponsor. We now have CAD $9,501 of the $30,000 goal or about 1/3rd of the goal to complete an Android plugin.

In order to get Android support added, we need 3 more corporate sponsors at the silver level (CAD $6500) or one at the gold (CAD $13,000) with a silver.

There are only 17 days to go.

If you believe Android is important to your project, please reach out to corporations and/or contribute yourself. With over 18,000 stars on the Slate repo, I find it hard to believe that it's not possible to raise funding to add Android support to this project which effectively adds the reach of Slate to almost 40% of the OS market.

I know there are some big corporations that use Slate so please reach out to them.

Add Android Support to Slate - KickStarter Campaign Now Live

[thesunny]

The good news is we are now at CAD $17,161 out of CAD $30,000.

The bad news is we have 6 days to go before the Kickstarter campaign is closed.

It's all or nothing so the funding has to complete.

The community asked to propose an offer to fund this project as open source and now it's up to you to get it funded. I've reached out to as many people as I can. We only need a little more to get this done! If you care about getting Android support in, please reach out to companies that could benefit from Android support. There are 18,400 stars but it means nothing if we can't reach the people that would benefit from this.

https://www.kickstarter.com/projects/sunnyhirai/add-android-support-for-the-slate-wysiwyg-editor?ref=user_menu

[thesunny]

Well everyone, this is the last chance. There is about 24 hours left.

We've raised CAD $18,461 out of CAD $30,000.

There is $11,539 left to go.

If you've been waiting to see if this gets funded without you, it's not! Now is the time to make the commitment. We need two silver sponsors to make this happen.

Last Chance at Android Support Kickstarter Campaign

[thesunny]

We made the fundraising goal on the Kickstarter!

A very generous sponsor added CAD $11,000 (over one third) of the goal and pushed us just over.

For privacy reasons, I don't want to name individual contributors, but thank you to everyone that contributed. We are slated to start in the second week of September.

[Slapbox]

Holy cow! I was bracing myself for disappointment this morning. This news blew me away. Very excited!

[rsdrsd]

This is very good news.

[thesunny]

Hi everyone,

We have an alpha version of a Slate Android Plugin available here:

https://github.com/wleroux/slate-android-plugin

It's not a final release by any means, but if you want to get started with it and report any issues not listed before, it could help us isolate bugs. It's posted under Wayne Leroux's GitHub account at the moment but may be moved in the future.

Notes about the alpha release:

• Requires the 0.59 version of Slate released a few days ago
• Only works on Android 10 (may work on Android 11)
• Continuous backspace has bugs
• Auto-Complete has bugs

Please post here if you have any issues

[rsdrsd]

Can someone maybe explain why an editor like ProseMirror doesn't have these problems on Android? Is it due to the architecture used or does ProseMirror have all kinds of hacks for Android?

[jljorgenson18]

Prosemirror still has Android problems and there are plenty of hacks inside of Prosemirror. You also lose consistent support for backspace and other keys because Android doesn't send keyCode information consistently. Prosemirror has some interesting ways of guessing that you did a backspace and then resetting the content based on the mutation.

Prosemirror does have a really awesome mutation registration system that is really nuanced and has many workarounds in place, but there are still many, many issues across device, OS version, webview version, and keyboard combinations. They are primarily just issues baked into contenteditable/Android though.

Prosemirror is still very, very good at Android compared to almost any other option and we still had to do insane workarounds to resolve issues.

[jljorgenson18]

Also, Prosemirror has really intelligent composition support where it tracks compositions and but then protects the composition text node until the composition is complete. The problem with compositions is that if you interact with composition text nodes at all, it can derail the whole composition (especially on Android). Prosemirror also works around various selection bugs that plague Chrome/Android Chrome with their state syncing system.

[coniel]

@thesunny Thanks a lot for this plugin!

I've run into an issue with soft line-breaks. Here's a simple reproduction (it simply prevents the default Enter key behavior and inserts \n as text instead). This somehow results in an infinite \n insertion loop when hitting Enter:

[thesunny]

Hi @coniel thanks for letting us know. These bug reports are really helpful for us.

I've mentioned this post to @wleroux who is the developer that's working on this version of the plugin.

[wleroux]

Thanks @coniel! I've published a new version of android-slate-plugin (version 0.0.3) to support soft line-breaks. :)

[swashata]

Hello,

When I select multiple words and try to delete, sometimes only the last letter is being deleted. Maybe it is happening because I am listening to selection and showing a popup toolbar. Please see this video. If needed, I can provide access to source code and a staging site to reproduce the issue.

https://youtu.be/OQQmAna6gWI

Using android-slate-plugin version 0.0.3

[coniel]

@wleroux Seems I forgot to thank you for the fix. So thanks! Can confirm that it works. I've been using the plugin since then and I've been seeing an error reported in my error logger, though I'm unable to reproduce it myself:

[knubie]

I completely missed the kickstarter for this. Is it too late to donate to the project?

[thesunny]

Hi @knubie 

Thanks for your offer to contribute to the KickStarter.

We've reached our donation target and adding any more to it really wouldn't change the outcome. We really appreciate the sentiment though.

One thing that will definitely help improve the success/quality of the project more than a monetary contribution is to try the plugin in your project and report any bugs. That will help make sure that the final release will be at as high a quality as it can be.

[bjgrosse]

I really would love to use Slate for my project, but Android support is a must. I've been testing out the plugin and am hitting several issues (besides the already mentioned bugs around continual backspace and selecting multiple words + backspace only removing the last character).

Here is my codesandbox: https://codesandbox.io/s/loving-hermann-6bdpq?file=/src/App.js

The first bug: I put my cursor in the middle of a word:3

I press enter, and I get this:

Continuing to press enter continues to generate duplicate characters:

The second issue, I take these steps:

1. Put curser at end of first line
2. Hit enter
3. type 'test'
4. hit enter
5. hit enter again (screenshot shows the serialized editor value):

1. hit backspace once
2. hit backspace again

1. Hit enter and get this error:

Now there is a new line space after the word 'test' but moving the cursor to it, triggers the above error again.

I hope this helps!

[wleroux]

Hello @bjgrosse , I could not replicate any of these issues. Which version of Android are you using? Which browser are you using?

[Slapbox]

@bjgrosse and also, what keyboard? I assume GBoard?

[dfee]

1. is the android plugin considered to be feature complete?

2. are you intending to merge that code into this repo, or maintain it separately? If maintain it separately, why?

[RobbyWH]

Hello i want to ask question, is the final release only supported for android 10 too?

[thesunny]

@dfee The Android Plugin is not considered feature complete at the moment. It is currently being tested against the latest version of Android. That said, it should be very good with one known bug of holding down the backspace key continually. This may be an issue that cannot be fixed and appears to be due to the fact that Android editing happens outside the JavaScript process. In other words, we can't seem to ever guarantee a state. If we read a document state (for example), it may have been modified by the time we reacted to it.

Ultimately, it would be great to have the plugin in slate proper. In the interim, we may keep it separate. The reason is that there is little expertise on the Android plugin and so it may be easy to break Android when there is a new Slate release. Unlike all the other OS and browser compatibility issues, Android takes a completely different approach to getting input and it does so, unfortunately, out of necessity. It is a separate implementation. By keeping Android as a plugin, we can lock the Android plugin to specific known good combinations of Slate and the Android plugin.

Over time, as expertise grows in the community, it may be reasonable to expect contributors to check for Android compatibility and run a suite of (currently) manual tests which hopefully can be automated at some point.

@RobbyWH Our deliverable with respect to the Kickstarter is the latest version of Android, but our hope is to deliver at least the latest 2 versions of Android. It may not be of that much additional benefit to go too far back in terms of Android support because it can add a lot of weight and complexity to the code. Generally speaking, the newer versions of Android have been easier to build for in my experience as they continue to improve the browser.

[wleroux]

@swashata , I was not able to reproduce your issue. If this is an unsupported version, you can find that information under ... > Settings > About Chrome? If it is a supported version, can you provide an environment where I can try it?

[wleroux]

@coniel I've updated the text diff algorithm to be more lenient. 0.0.4 should resolve your issue. What might be happening is the editor is trying to modify non-editable text within your editor, and the slate-android-plugin is not detecting the corresponding leaf node as a result.

[swashata]

Hi @wleroux Thank you for your attention. It is indeed an unsupported version (sorry I missed that).

Application version: Chrome 86.0.4240.198
Operating system: Android 11; IN2021 Build/RP1A.201005.001

The keyboard I used is GBoard. I will get back to you with a reproducible environment where you can try.

[markgrancapal]

Just want to report that I encountered the same issue @swashata posted here.

Application version: Chrome 87.0.4280.66
OS: Android 10

This may be an issue that cannot be fixed and appears to be due to the fact that Android editing happens outside the JavaScript process. In other words, we can't seem to ever guarantee a state. If we read a document state (for example), it may have been modified by the time we reacted to it.

@thesunny is there no solution for this continues backspace bug?

[wleroux]

@markgrancapal I've recently acquired an Android 11 device to investigate @swashata's issue, haven't tried to reproduce it there yet. My test environment is very similar to yours (Chrome 87.0.4280.101, Android 10). Are you using the default keyboard or any other customizations?

RE: backspace issue... The issue with the backspace has to do with how Android handles contenteditable and Slate's cursor synchronization code. Android modifies the cursor out-of-band from javascript, which makes it impossible intercept the changes before Slate's synchronization code. The approach we took was to avoid modifications to the Slate codebase entirely.

My current hypothesis on the issue is that in attempting to override Slate's default handling of composition events, we introduce these other issues. There is no way to override Slate's default handling of composition events without either preventing default or stopping propagation of these events, however without overriding the composition events, it doesn't work properly at all. I am working on validating that hypothesis.

[markgrancapal]

@wleroux Yes I'm using the default Keyboard GBoard.

That is quite complicated, handling the backspace issue. I hope you guys will find a solution on this. I'll test it again and send some feedback when you release a new version

[thesunny]

Hi Everyone,

It's the end of the year and, sadly, it looks like we aren't able release a complete version of the Slate Android plugin in time for the December 2020 goal. Wayne is continuing to work on the plugin and we will be focusing all of his available time on the Slate Android plugin until it is complete.

I was hoping to not have to get involved but if we aren't able to complete in a reasonable time, I'll consider my jumping back in as described in the Kickstarter.

Thank you for your patience. Android support will get done. As mentioned, it is something we need in our projects as well so we are strongly motivated to complete.