Wp-calypso: Gutenberg: Opt-in calls from mobile can sometimes fail.

[kwight]

Calls to the opt-in API endpoint from mobile apps can fail, if many calls are made at once. No errors are reported from the POST call, but manual verification shows that some sites will not be updated.

As described by @etoledom :

An example:

• We have 5 sites set with mobile_editor: aztec
• On a for loop, we send 5 post requests to set mobile_editor: gutenberg. These requests are being executed in parallel.
• All requests succeeded (no error was received on client side)
• On the response body, we see that (as an example) 2 out of the 5 sites says mobile_editor: aztec, given that the request was to be set to gutenberg.
• Checking the user settings, we see that effectively, those 2 sites are still set to mobile_editor: aztec

The number of sites failing are kind of random, but we noticed that less of them fail if we add a small delay in between the calls.

See: paAmJe-xU-p2 (#comment-1719)

[daniloercoli]

I've just experienced the same issue on the Android app. The app made a series of POST requests but many of them have problems. The HTTP response code of those calls were 200, but the value returned by the server is kind of unexpected.

Below are the details of the first POST request made by the app (first request of a series, and the first to fails. That's weird since the server should not be under loading/race condition at this point):

:method: POST
:path: /wpcom/v2/sites/15592836/gutenberg/?_locale=it_IT
:authority: public-api.wordpress.com
:scheme: https
user-agent: Mozilla/5.0 (Linux; Android 7.1.2; Nexus 5 Build/NJH47F; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/69.0.3497.109 Mobile Safari/537.36 wp-android/12.9
authorization: Bearer AUTHORED
content-type: application/json; charset=utf-8
content-length: 42
accept-encoding: gzip

{"platform":"mobile","editor":"gutenberg"}

and this is the response of the server:

:status: 200
server: nginx
date: Wed, 31 Jul 2019 09:33:28 GMT
content-type: application/json; charset=UTF-8
vary: Accept-Encoding
set-cookie: _wpndash=c7d7bd6f755; expires=Thu, 30-Jul-2020 09:33:27 GMT; Max-Age=31536000; path=/; domain=.wordpress.com
set-cookie: recognized_logins=hsfBWYzQV-ZwSwhE1WCyiHvyD; expires=Sat, 28-Jan-2023 21:33:27 GMT; Max-Age=110376000; path=/; domain=wordpress.com; secure; HttpOnly
x-hacker: Oh, Awesome: I/Opossum
x-robots-tag: noindex
link: <https://public-api.wordpress.com></https:>; rel="https://api.w.org/"
x-content-type-options: nosniff
access-control-expose-headers: X-WP-Total, X-WP-TotalPages
expires: Wed, 11 Jan 1984 05:00:00 GMT
cache-control: no-cache, must-revalidate, max-age=0
allow: GET, POST, PUT, PATCH
access-control-allow-headers: Authorization, Content-Type
content-encoding: gzip
x-ac: 2.mxp _dca 
strict-transport-security: max-age=15552000

{"editor_mobile":"","editor_web":"classic"}

The JSON response for editor_mobile is still empty. If you go to my RC card, there is no entry for the site with ID 15592836 - eritreocazzulati.wordpress.com.

Also, you can see from the log above the response is missing of the opt_out link.

The 2nd request made by the app failed to set the editor.

The 3rd request instead worked without problem and the response was the following:

:status: 200
server: nginx
date: Wed, 31 Jul 2019 09:33:29 GMT
content-type: application/json; charset=UTF-8
vary: Accept-Encoding
set-cookie: _wpndash=c7dffba5cb0f755; expires=Thu, 30-Jul-2020 09:33:28 GMT; Max-Age=31536000; path=/; domain=.wordpress.com
set-cookie: recognized_logins=f_nwr-Xo8NzleJ3HNJLdj6eG4t95BekSEmUrlsvSK3cjXLZjvzJCikjnUu8%3D; expires=Sat, 28-Jan-2023 21:33:28 GMT; Max-Age=110376000; path=/; domain=wordpress.com; secure; HttpOnly
x-hacker: Oh, Awesome: I/Opossum
x-robots-tag: noindex
link: <https://public-api.wordpress.com></https:>; rel="https://api.w.org/"
x-content-type-options: nosniff
access-control-expose-headers: X-WP-Total, X-WP-TotalPages
expires: Wed, 11 Jan 1984 05:00:00 GMT
cache-control: no-cache, must-revalidate, max-age=0
allow: GET, POST, PUT, PATCH
access-control-allow-headers: Authorization, Content-Type
content-encoding: gzip
x-ac: 2.mxp _dca 
strict-transport-security: max-age=15552000

{"editor_mobile":"gutenberg","editor_web":"classic","opt_out":"https:\/\/public-api.wordpress.com\/wpcom\/v2\/sites\/79289474\/gutenberg?editor=aztec&platform=mobile"

It does contain the opt_out link.

Another question, could it be a race or caching problem, with more requests that try to update a user's attribute, and introduce conflicts?

[daniloercoli]

I also noticed one weird thing that happens when the mobile app do many gutenberg POST calls.

On some "random" sites, the value of the editor for the web option is set after the upgrade. To replicate the problem I cleared all the editor preference from my RC card and let the app do many POST calls.

Curious to understand why it is happening, no?

[daniloercoli]

I've added some logging to the code, and it seems that the wrong responses happen when 2 POSTs hit the server at the same time.
My investigation is far to be complete, but the log below may help a bit.

[01-Aug-2019 09:05:02 UTC] WPCOM_REST_API_V2_Endpoint_Site_Gutenberg-> set_state called on  11400796 platform mobile editor gutenberg
[01-Aug-2019 09:05:02 UTC] WPCOM_REST_API_V2_Endpoint_Site_Gutenberg-> set_state called on  15835028 platform mobile editor gutenberg
[01-Aug-2019 09:05:02 UTC] opt-in.php -> set_editor -> updated editor for  11400796 platform mobile editor gutenberg
[01-Aug-2019 09:05:02 UTC] opt-in.php -> set_editor -> updated editor for  15835028 platform mobile editor gutenberg
[01-Aug-2019 09:05:02 UTC] WPCOM_REST_API_V2_Endpoint_Site_Gutenberg -> set_state -> response for  15835028 platform mobile editor gutenberg 
WP_REST_Response Object
(
    [links:protected] => Array
        (
        )
    [matched_route:protected] => 
    [matched_handler:protected] => 
    [data] => Array
        (
            [editor_mobile] => gutenberg
            [editor_web] => classic
            [opt_out] => https://public-api.wordpress.com/wpcom/v2/sites/15835028/gutenberg?editor=aztec&platform=mobile
        )
    [headers] => Array
        (
        )
    [status] => 200
)
[01-Aug-2019 09:05:02 UTC]  WPCOM_REST_API_V2_Endpoint_Site_Gutenberg -> set_state -> response for   11400796 platform mobile editor gutenberg WP_REST_Response Object
(
    [links:protected] => Array
        (
        )
    [matched_route:protected] => 
    [matched_handler:protected] => 
    [data] => Array
        (
            [editor_mobile] => 
            [editor_web] => classic
        )
    [headers] => Array
        (
        )
    [status] => 200
)

[koke]

Simplifying, what the backend code is doing is this:

$editors = get_user_attribute( $user_id, 'editors' );
$editors[ $blog_id ][ $platform ] = $editor;
update_user_attribute( $user_id, 'editors', $editors );

This is not an atomic operation, so if two requests arrive at the same time, the second will overwrite the changes of the first.

For instance, if we set gutenberg for blogs 1 and 2:

• 1: $editors = get_user_attribute( $user_id, 'editors' ) -> []
• 2: $editors = get_user_attribute( $user_id, 'editors' ) -> []
• 1: $editors[1]['mobile'] = 'gutenberg' -> [1 => ['mobile' => 'gutenberg']]
• 2: $editors[2]['mobile'] = 'gutenberg' -> [2 => ['mobile' => 'gutenberg']]
• 1: update_user_attribute( $user_id, 'editors', [1 => ['mobile' => 'gutenberg']] )
• 2: update_user_attribute( $user_id, 'editors', [2 => ['mobile' => 'gutenberg']] )
• Final result: [2 => ['mobile' => 'gutenberg']]

[daniloercoli]

Thanks @koke for recapping the whole problem. I also think there is also something more happing here, since I see some web values set after those mobile calls.
Maybe unrelated, but wondering who may set those web preferences, considering that i'm not doing changes on the web.

[koke]

There are some cases in the API where the if the blog has a enable-gutenberg flag, trying to read the value for the web will store the default in the user preferences.

[kwight]

Thanks @koke , that sounds like the issue.

There are some cases in the API where the if the blog has a enable-gutenberg flag, trying to read the value for the web will store the default in the user preferences.

Correct, that was the best we could do to maintain editor selection for users coming back to a site after a period, where that site had the blog sticker from the original implementation. We'd like to rip that out at some point down the road.

So if I understand correctly, when the app upgrades, it sends a pile of API POST requests for all of the user's sites all at once? Options could be creating an async job server-side (which is understandably a pain), or could they be done from the client in serial one-after-the-other instead? I'm guessing the great majority of users don't have more than a handful of sites to iterate through?

[kwight]

@gwwar made some points in p9ugOq-Bw-p2 that limiting individual calls from clients could be preferable; maybe some sort of flag representing that app setting could work. (Let's continue convo in that P2 post.)

[mmtr]

Closed by D31159-code and D31161-code.

We finally added a new /me/gutenberg endpoint to the API that will allow to set the editor for all the sites in one single request.