Wordpress-ios: Technical solution to trigger new actions when the device goes online

[diegoreymendez]

Description:

Certain operations may need to be triggered when the device goes back online (after being offline). The scope of this issue is to evaluate our current level of support for this and to propose the changes needed to make this happen.

Below is a list of issues that are probably affected by this feature.

Affected Issues:

https://github.com/wordpress-mobile/WordPress-iOS/issues/11431
https://github.com/wordpress-mobile/WordPress-iOS/issues/11449
https://github.com/wordpress-mobile/WordPress-iOS/issues/11425
https://github.com/wordpress-mobile/WordPress-iOS/issues/11699

Corresponding Android Issue:

https://github.com/wordpress-mobile/WordPress-Android/issues/9803

Issues spawned by this discussion:

https://github.com/wordpress-mobile/WordPress-iOS/issues/11653
https://github.com/wordpress-mobile/WordPress-iOS/issues/11694

[diegoreymendez]

As discussed privately via DM with @SergioEstevao, I don't think there's any way in iOS for us to do background uploads when the device goes back online, which is something that's being discussed in the Android issues.

My suggested triggers are:

• When the App starts up / when the App comes to the foreground.
• When we detect there's a connection, after being offline.

These are some other conditions we can check for before attempting to upload anything:

• Low power mode is OFF.
• Ideally the device should NOT be roaming. This is something we'll have to explore further as I'm not aware of any mechanism to detect this easily in iOS.

It could also make sense to have settings for these.

Once any of the triggers is activated, we should query our DB for posts that are not synchronized and retry.

[SergioEstevao]

Ideally the device should NOT be roaming. This is something we'll have to explore further as I'm not aware of any mechanism to detect this easily in iOS.

If roaming is not an option, maybe have a setting to say if we want to allow it only when connected to WIFI?

[yaelirub]

As discussed privately via DM with @SergioEstevao, I don't think there's any way in iOS for us to do background uploads when the device goes back online, which is something that's being discussed in the Android issues.

Interesting. What about using something like RequestRetrier ?

[SergioEstevao]

The request retrier could be used, for lower level retries on spotty networks, but I do believe we need an higher level solution to reason about the state of the media objects as a group inside a post.

So for example if a post has already two media objects uploaded but two others failed, it will need to retry those two, await for them to complete, update the post and the retry to upload the post.

Sent from my iPhone

On 7 May 2019, at 22:37, yaelirub [e-mail] wrote:

As discussed privately via DM with @SergioEstevao, I don't think there's any way in iOS for us to do background uploads when the device goes back online, which is something that's being discussed in the Android issues.

Interesting. What about using something like RequestRetrier ?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[diegoreymendez]

I think retrying requests is more of a solution for them moment when the request fails. For longer term offline interruptions I agree with @SergioEstevao and I think the trick is in having a local data structure that allows us to query exactly what needs to be uploaded.

[SergioEstevao]

I think the trick is in having a local data structure that allows us

I do believe the current CoreData information in the Post and Media objects have enough information for us to reason on what to do next in terms of uploads.

[diegoreymendez]

I don’t think we need big changes but I do want to analize the different scenarios and make sure we have a proper way to represent each one. I don’t think we have a proper representation for multiple local edit sessions yet. I’ll write more about this tomorrow.

[shiki]

I don't think there's any way in iOS for us to do background uploads when the device goes back online

We probably don't need to anyway. Considering Reachability is not accurate, we can instead observe different events and then try to upload even if the device is offline.

App in background

There are a few options that I know of but haven't tried. The linked documentation pages mention "download" but I believe uploading is possible.

• Using Push Notifications to Initiate a Download - when the user receives a push notification, we can try to quickly upload pending posts. I'm not sure about this since it may be irresponsible for us to do something unrelated to the push notification.
• Significant-Change Location Service - I believe this is how Dropbox automatically starts uploads. I noticed it happens when I arrive at home or go to a known location with wi-fi. The caveat is, we have to ask for location permissions.
• Background Fetch - triggered based on the OS' whim. This may be the best option. It's not reliable but it's better than nothing when the app is in the background.

App in foreground

• Observing Reachability changes
• When the app is placed in the foreground.

[diegoreymendez]

I think we should split this discussion. The "App in foreground" scenario is pretty straightforward so I've opened an issue to implement those triggers.

We still need to figure out what to do with the "App in background" scenario, so let's focus the discussion on that part.

[diegoreymendez]

This graphic shows an initial proposal of how we could handle failed uploads in iOS.

• The yellow round shapes on the left are the triggers.
• The green squares are classes.
• The violet squares are protocols.

One thing this mechanism doesn't really take into account, though, is concurrency. If something is being uploaded through a direct request, and one of the triggers starts FailedUploadManager, we need to make sure we don't start a different upload for it.

This is a problem with services, because they are instantiated each time you need them.

One thought on my mind is that this is less of a problem with MediaCoordinator since it's a singleton and knows about all media uploads.

This makes me think we may want to consider having something like a PostUploadCoordinator to take care of the post uploads instead of using our traditional Service classes.

They two persons that I know that have the most knowledge about Service classes and MediaCoordinator are @astralbodies and @SergioEstevao. Can I ask you to share your thoughts on concurrent uploads?

Also CC @maxme in case you can comment if concurrency is considered / dealt with in Android.

[diegoreymendez]

Diagram Update

While reviewing my work a bit today I realized a few things.

1. Failed uploads and regular uploads are the same thing for the purpose of delivering our data to the servers. We just need to make sure there are coordinators to handle all uploads.
2. There's a PostCoordinator which is more appropriate for this goal than trying to use PostService directly.

This is my latest diagram:

Querying for objects to upload:

Additionally, we need to make sure these coordinators have a very clear understanding of what objects need to be uploaded, and which ones don't. A simple query should provide us with this information.

For both posts and media we have these statuses:

From AbstractPost.h:

typedef NS_ENUM(NSUInteger, AbstractPostRemoteStatus) {
    AbstractPostRemoteStatusPushing,    // Uploading post
    AbstractPostRemoteStatusFailed,      // Upload failed
    AbstractPostRemoteStatusLocal,       // Only local version
    AbstractPostRemoteStatusSync,       // Post uploaded
    AbstractPostRemoteStatusPushingMedia, // Push Media
};

From Media.h:

typedef NS_ENUM(NSUInteger, MediaRemoteStatus) {
    MediaRemoteStatusSync,          /* Post synced. */
    MediaRemoteStatusFailed,        /* Upload failed. */
    MediaRemoteStatusLocal,         /* Only local version. */
    MediaRemoteStatusPushing,       /* Uploading post. */
    MediaRemoteStatusProcessing,    /* Intermediate status before uploading. */
    MediaRemoteStatusStub,          /* We only have the mediaID information from the server */
};

For starters I believe we can query the DB for objects with status == .failed.

Some Space for Improvement:

This is a bit beyond scope, and I want to separate concerns a bit to simplify things.

That said, I think we have a bit of an overcomplicated architecture around sync status which could be made much better.

The status above aren't great as they are. Some of those status should be transient, and their lifecycles should be tied to the coordinators.

I believe the status we want persisted across App runs are .local, .sync .failed and probably .stub (although I'm not sure what this last one is).

The status .pushing and processing however should be 100% transient and should die with the App.

A hint of this being a problem can be found here.

[SergioEstevao]

The status .pushing and processing however should be 100% transient and should die with the App.

I think the .pushing status can be saved, because when you have a background upload the app can be killed and the upload still going on in the background.

The .stub status exists, because sometimes we can have reference to Media objects, that are still not fully fetched from the remote server. For example you can download a post, get a mediaId for it's featured image but you don't have the full media information. So at that point you have Media object that is a stub.

[yaelirub]

I believe the status we want persisted across App runs are .local, .sync .failed and probably .stub (although I'm not sure what this last one is).

Post preview PR introduces a new post status AbstractPostRemoteStatusAutoSave which I assume we would want to persist as well.

[diegoreymendez]

I think the .pushing status can be saved, because when you have a background upload the app can be killed and the upload still going on in the background.

But when the App launches the coordinator should be aware of background uploads (for concurrency purposes), meaning the .pushing status can still be transient and handled directly by the coordinator.

In other words it should be kind of a dynamic property instead of something we store statically.

[diegoreymendez]

Post preview PR introduces a new post status AbstractPostRemoteStatusAutoSave which I assume we would want to persist as well.

That's correct, I believe.

[maxme]

Also CC @maxme in case you can comment if concurrency is considered / dealt with in Android.

I don't think we should implement concurrent uploads. On mobile we're likely limited by the bandwidth, so we won't gain much by having concurrent uploads and it will complexify what we use to manage the queue/UI in the app.

[diegoreymendez]

I don't think we should implement concurrent uploads. On mobile we're likely limited by the bandwidth, so we won't gain much by having concurrent uploads and it will complexify what we use to manage the queue/UI in the app.

Well... no concurrency is a perfectly valid answer. Thanks!

That said, I was wondering if we do anything on Android to prevent the same post / media item from being queued twice for upload.

[diegoreymendez]

I created another issue from this discussion to implement the Uploader protocol in PostCoordinator and MediaCoordinator.

I also updated this issue's description to include a list of the new issues we create from this discussion.

[maxme]

That said, I was wondering if we do anything on Android to prevent the same post / media item from being queued twice for upload.

Yes, In FluxC, when we add a post to the queue, we check if there is already a post with the same id queued, if that's the case, we update the queue item (only associated media ids are updated because we need to know which media have been uploaded or waiting in queue before publishing a post).

[diegoreymendez]

Subscribing @wordpress-mobile/ravenclaw

[diegoreymendez]

I'm closing this issue. We already have a technical direction to start working. If there's any other more specific discussion we can tackle it separately.