Wordpress-android: Offline Support: Local drafts aren't synched when going back online.

[diegoreymendez]

Description

Local drafts with changes aren’t uploaded to the server after a connection is re-established

Opening and closing the post or pulling to refresh the post list don't force the post to sync either.

There's also no "Update" action when going into posts that were created while offline.

Matching iOS issue

https://github.com/wordpress-mobile/WordPress-iOS/issues/11449

Tracking

| Description | Status |
|--------|-------|
|

Syncing after leaving Editor.

After creating a new post while offline. When you come back online and open the post, then back out of the post, the local changes are not synced.

| ❓Could not reproduce anymore. |
|

Swipe to refresh.

In this gif, local changes were saved while offline and I expected pull to refresh to start syncing the changes:

| ✔️ Done in #9774 |
|

Different Editor actions.

And this gif might be a new issue. Notice how both of these posts are drafts, but they have different actions in the editor (Publish vs Update).

| ✔️Will be handled in #9612 |
| Auto-upload for Pages. | ✔️ Will be handled in #9851 |
|

Media in draft posts do not get automatically uploaded.

Tapping the _Retry_ button after works though.

| ✔️Will be handled in #9804 |
|

Auto-upload of Local Changes.

| ✔️ Will be handled in #9846 |

[shiki]

@diegoreymendez It looks like this is the same as iOS. Do we have an issue created for iOS? Can't seem to find it.

[diegoreymendez]

@shiki - I think the main problem with this issue is that the description is a bit confusing. I was assuming this was specifically about the list of posts, which is refreshing automatically for me when the connection comes back online.

Also "upload" and "sync" have different meanings. I'll try to clarify the description.

[oguzkocer]

Here is what I have been thinking about syncing in general:

1. Either have 2 separate DBs one representing the last known remote state and one representing the local state or separate the posts table into at least 2 tables again one representing the remote state and one representing the local state
2. Have a reference for which remote posts a local change is applied on. This is not only just the remoteId of a post, but also possibly a hash of the remote post that's changed. This will give us a way to know whether changes were made to a post in both local and remote at the same time.
3. For conflict resolution, we can let the user decide which version to keep with the ability the diff of the changes. (of course we don't have to)
4. If the user changes the remote version of a post, we can even create a revision with the local changes for safe keeping. This way a change will never be lost and if the user accidentally accepts the remote version they can just revert it using the revision.
5. Most importantly: Separate the remote post status from the local one. We should have our own states which should make it much easier to sync changes with remote.

[diegoreymendez]

1. Either have 2 separate DBs one representing the last known remote state and one representing the local state or separate the posts table into at least 2 tables again one representing the remote state and one representing the local state

This is interesting for me because I feel a lot of the complex issues we've had with posts (at least in iOS) are caused by us not differentiating remote and local state well enough.

I'd feel the 2 tables approach is nicer if we want cross references, but I'd also like to look into this a bit more on my end (and hear other thoughts).

In iOS we already have something like this, but the way it's implemented is extremely confusing IMHO. We have original and revision posts (not the revisions you know though, it's a bit of a separate thing... like a working copy of the post) in the same table - but the main issue is that the meaning of what these revisions represent isn't very clear and have different interpretations depending on who you ask.

1. Have a reference for which remote posts a local change is applied on. This is not only just the remoteId of a post, but also possibly a hash of the remote post that's changed. This will give us a way to know whether changes were made to a post in both local and remote at the same time.

This is one good way to tackle the conflict resolution.

A locally-calculated hash of the post item could be enough for this, I believe?

1. For conflict resolution, we can let the user decide which version to keep with the ability the diff of the changes. (of course we don't have to)

Agree.

1. If the user changes the remote version of a post, we can even create a revision with the local changes for safe keeping. This way a change will never be lost and if the user accidentally accepts the remote version they can just revert it using the revision.

I agree with this as well. I think revisions can make it so much easier for us to still save the local changes somewhere.

1. Most importantly: Separate the remote post status from the local one. We should have our own states which should make it much easier to sync changes with remote.

I agree, let's start thinking more in this direction.

Also, one thing that worries me a bit is how all of this is going to work for non-WP.com sites. If we can manage to only have local changes (I don't know yet), this may turn out to be a non-problem.

[oguzkocer]

I'd feel the 2 tables approach is nicer if we want cross references, but I'd also like to look into this a bit more on my end (and hear other thoughts).

I agree, this is the more performant option and I'd start with it. However, I wanted to mention that because if we need more tables at one point we might say that we are OK with the minor performance hit to have an easier differentiation.

A locally-calculated hash of the post item could be enough for this, I believe?

Ideally we'd have the remote hash so that we can send that hash in push post network action so the endpoint can deny changes if we don't use the latest hash. This will make sure that we are not overriding changes in remote accidentally. I think a modified date would also work, but it wouldn't be as good and the current modified date is not always updated. So 🤷‍♂️

Also, one thing that worries me a bit is how all of this is going to work for non-WP.com sites. If we can manage to only have local changes (I don't know yet), this may turn out to be a non-problem.

I think the general approach would work for both self-hosted and REST API sites, however I might be missing something. Not all of it will work of course, but I think that's OK. The idea is to make the constraints visible for the developer for each action so that they can take the "best" approach within the given constraints.

[malinajirka]

I like your suggestions @oguzkocer!

1. Either have 2 separate DBs one representing the last known remote state and one representing the local state or separate the posts table into at least 2 tables again one representing the remote state and one representing the local state

with the minor performance hit to have an easier differentiation.

I'd definitely vote for clearer solution over a theoretically more performant solution. I think Sqlite is quite fast and unless we do some crazy things or we have tens of thousands of record I don't think there will be any significant impact.
I don't have a good experience working with a complex databases on the client so whether it's better to have one vs more dbs/tables really depends on the specific solution.

1. Have a reference for which remote posts a local change is applied on. This is not only just the remoteId of a post, but also possibly a hash of the remote post that's changed. This will give us a way to know whether changes were made to a post in both local and remote at the same time.

A couple of questions regarding the solution with hashes in the API.

• Would adding a field to the table on the remote(server) be a breaking change? As it'd change the hash and as result would mark all locally changed posts on Android clients as conflicts, right?

• XMLRPC sites already support last modified date (even though it doesn't work in 100% cases). AFAIU we'd need to solve the same problem twice - once for self-hosted sites (modified date) and once for wpcom sites (remote hash)? Or am I missing something?

1. For conflict resolution, we can let the user decide which version to keep with the ability the diff of the changes. (of course we don't have to)

That'd be awesome - we might be able to reuse the DiffView we use for revisions.

1. If the user changes the remote version of a post, we can even create a revision with the local changes for safe keeping. This way a change will never be lost and if the user accidentally accepts the remote version they can just revert it using the revision.

I'm not sure how revisions work in detail - there might be some issues with the diff if we added "artificial" revision, but it's another topic and I like the idea;).

1. Most importantly: Separate the remote post status from the local one. We should have our own states which should make it much easier to sync changes with remote.

💯 👍

[megsfulton]

Apologies for the vague description, these were hard to describe.

Local drafts with changes aren’t uploaded to the server after a connection is re-established, Opening and closing the post or pulling to refresh the post list don't force the post to sync either.

In this gif, local changes were saved while offline and I expected pull to refresh to start syncing the changes:

This gif creates a new post while offline. When you come back online and open the post, then back out of the post, the local changes are not synced.

There's also no "Update" action when going into posts that were created while offline.

And this gif might be a new issue. Notice how both of these posts are drafts, but they have different actions in the editor (Publish vs Update).