Web-stories-wp: Status Check: Add REST API endpoint

[swissspidy]

Feature Description

We're seeing reports from users where they create a story, and upon saving get greeted with error messages because saving failed due to the REST API not being available.

The problem is that they might not be saving for quite some time, so it's really frustrating to get error messages after an hour or so and potentially lose all your work.

What If upon loading of the editor/dashboard we perform some simple REST API request and see if the response is OK. If it isn't, then the REST API is likely not available, and we can display a warning message to the user.

~We could probably just re-use \WP_Site_Health::get_test_rest_availability() for this. Alternatively we could do the same thing on the client-side.~ -> WAF detection is only possible on client-side.

Downside: this is another HTTP request that might be unnecessary for the majority of users.

Alternatives Considered

Better documentation that the REST API is required

Additional Context

---

_Do not alter or remove anything below. The following sections will be managed by moderators only._

Acceptance Criteria

Implementation Brief

• When loading dashboard/editor...
  
  
  
  • Create a temporary story in PHP with auto-draft status, using demo content for post_content field, just so we have a WP_Post object to work with.
  
  
  • Provide this story object via an inline script
  
  
  • On the client side, generate the story's HTML markup using getStoryMarkup()
  
  
  • Send a POST request containing this full markup to /wp-json/web-stories/v1/web-story/<id>
  
  
  • -> check status code, response content, etc. to see if this was successful or not
    
    Exact checks TBD
  
  

Open questions:

• Create new web-stories/v1/status-check REST API endpoint for this?
• Also send a GET request?

[swissspidy]

@o-fernandez @choumx @ernee Curious to hear your thoughts on this idea

[spacedmonkey]

There are two cases here we need to consider.

1. There is some sort of http request error in the browser and the communicate error happens.
In this case, the above, health check would work.
2. There is an auth error or the REST API is disabled / locked down.
I have seen some larger publishers and enterprises lock down on the REST API. Either disabling it completely or locking down just some allowed routes. In this case, we should use these preloads in edit-stories and dashboard to do some error handling. If either of these requests return empty or a WP_Error, display an error message to the user instead of rendering the editor / dashboard.

We could also generally improve the error handling on saving of stories. Take this for example.
https://github.com/google/web-stories-wp/blob/9cd13ee04a8ec6240017e2fdd513fe3b45720e23/assets/src/edit-story/app/story/actions/useSaveStory.js#L90-L92

At this point in the code, we have the status that was return from the request. We should show different errors here. Take for example, a 403 auth error, could show a permission error etc. Also see https://github.com/WordPress/gutenberg/pull/15657 for more context on this.

[o-fernandez]

This is great. Open to options, but these improvements to let users know proactively of issues is very important. Although it is, indeed, an enhancement, I'd prioritize this as a P1/P2 bug given the user frustration.

[swissspidy]

I don‘t think simply using the Site Health check that way works, actually. For the reports we received the Sit Health check was green. Also, while strengthening the preloading might make sense in general, it‘s not the root of error. It doesn‘t go over HTTP anyway.

If the REST API request is blocked by a WAF, neither will catch this. We need an early, simple, client-side check.

[spacedmonkey]

As far as I understood, the site health check was a client side check, but I could be wrong. That might be the state that they were talking about when we built some of the stuff and was never implemented.

For that what is worth, this should likely be two tasks.

1. Client side check that http requests to the REST API are functioning. P1/P2
2. Error handling and strengthening of REST API that are preloading. P3.

[swissspidy]

Yup, makes sense. Feel free to create an issue for 2), as I opened this one specifically for 1).

[spacedmonkey]

Related: https://github.com/WordPress/wordpress-develop/pull/381 #48105

[choumx]

SGTM. Should be able to differentiate unrecoverable errors via HTTP status code e.g. 5xx.

P2 tangent: for transient/network errors, it might be nice to have an auto-save indicator in the UI that shows an error icon (e.g. red "!") when it fails.

[spacedmonkey]

This ticket does not seem like it ready for sprint. There need to UX for what happens if the API is not available. I think we need different messaging for different status code errors. For example, a

• 50x error, should be "Error in connection, please try again"
• 403 error should be "Authentication error
• Timeout or refused request should be "Communication error, please contact adminstator to fix".

I think this should be moved back to UX / product.

[swissspidy]

I don't think we need to completely block this on UX. We should split this up though:

• Design logic for status checks, review & implement (can behind a feature flag)
• Surface findings to the user (that part needs UX)
• New ticket for better error messaging in the editor (as requested above)

The design part is not blocked by UX

[swissspidy]

We created #4838 as an epic to keep track of all these things, and #4847 specifically for the UX part.

This ticket here is specifically for building the client-side infrastructure for this kind of status check.

[spacedmonkey]

So we want to add a health check api call once the editor is fully loaded, to check that browser can connect api. At what point can be mark the editor as done / rendered that will also not affect user interacting with the editor. I would love the thoughts of @barklund and @miina on from the Workspace pod.

[swissspidy]

I suppose it could be as simple as adding a new <StatusCheck> component here:

https://github.com/google/web-stories-wp/blob/8e18351b72654ee89a027607f8625f34e15c4d25/assets/src/edit-story/app/index.js#L70-L77

The component could perhaps work like this (simplified):

function StatusCheck() {
  const [errors, setErrors] = useState([]);

  useEffect(() => {
    async runChecks() {
      // ... run checks
      await doSomething();
      setErrors(...)
    }

    runChecks();
  }, [])

  if (!errors)
    return null;

  return (
    <Modal ...>
     ...error messages
    </Modal>
  );
}

This should run things without blocking rendering of the rest of the editor.

The error message can be super simple until #4847 is implemented. And until we have #4847 it's probably best to keep this behind a feature flag.

[bmattb]

@o-fernandez it was flagged in backlog grooming today that this ticket will not be completed by the end of S39 and will carry into S40

[spacedmonkey]

A couple more options to consider.

1. We simply remove the preload api request for stories. That way, we require a REST API call before the editor can start. We then can handle errors from there.
2. My original idea was to add a new health check REST API and call this async in the editor. This call would be a POST request and would response quickly. ( I think this is the best option ).

Do we even need a status check in the dashboard. The only api requests that are done are to get second pages and or when settings are updated.

[swissspidy]

We simply remove the preload api request for stories. That way, we require a REST API call before the editor can start. We then can handle errors from there.

Sounds like the easy option. But I don't think we should do this. The preloading makes for much improved perceived performance.

My original idea was to add a new health check REST API and call this async in the editor. This call would be a POST request and would response quickly.

Makes sense, just want to note that this REST API call would need to be very similar to an actual API call, e.g. send HTML markup to see if it's blocked by a firewall. Also, a second GET request would also be good.

Do we even need a status check in the dashboard. The only api requests that are done are to get second pages and or when settings are updated.

Context: We had a few reports for the dashboard, for example when users tried to create a story from a template (because the HTML markup is blocked).

At very least, the health check should be done in a way that it _could_ be used in both the editor and dashboard.

[spacedmonkey]

I have a work in progress PR. This simply sends a REST API post request to new api and if that request fails, it shows a message. I have a little video demo of it working. See demo.

I need to feedback from @o-fernandez / @samitron7, on what exactly should happen if the status/health check fails. How do we warn the user that the editor will not work for them?

[o-fernandez]

@spacedmonkey following up from our conversation in sprint planning - can you elaborate a bit on what are the reasons why the API post request may fail? Is it always the same reason, or is there a chance for interim issues that would be solved by just retrying?

Will any of these be possible:

• WP Hosting company may be blocking the API requests
• Other WP plugin interference?
• 3P service interference?
• Interim server issue?

Can you let me know and see if it's possible that it's a variety of things? If so, we may want to ask users to reach out to our support forum to try to help them sort it out, until we can provide a better guidance.

If there can be interim failures that resolve themselves, we'd want to minimize the chance of alerting users about this. So, we may want to try a few times prior to erroring out.

[spacedmonkey]

@o-fernandez

I responded to this 3 days ago. I have no idea where my answer went. I am a little mad, as it was a lengthy and detailed reply :(
I will try and remember what I said before.

Reasons why an update / post request might fail.

• Hosting provider blocks the request. This could be at the server, CDN or firewall levels. This is normally becuase they have not whitelisted a url path, pattern. But as discovered in hosting envoriments, posting raw html is also blocked. See for #4805 for more details.
• Hosting provide / developer has block rest API requests at a code level. Some publishers are worried about people selling data so lock down all rest api requests.
• User loses internet connection while opening the browser. Internet may drop out or the editor maybe cached in a browser.
• There is a PHP fatal error on the server, that means that REST API returns a 500 error instead of valid response.
• There is a database connection or timeout error that also results in the PHP error being returned.

Listed above are the most likely error statements, but there will be many more smaller edge cases. Hope this answers the question, sorry for the late reply, it was not by design, but a github failure.

[o-fernandez]

This is super helpful, @spacedmonkey, and no worries about the delay.

Given the various issues, would it be best to first make a few checks in intervals (e.g., several checks over a 5-min window) and only show a failure to the user if we REST API calls consistently fail? That would hopefully help avoid showing an error for transient issues.

For those that do fail, we could then give them an actionable path to figuring things out. One option would be to ask them to file a support ticket where we can request more information to try to narrow down what the issue may be and guide them on next steps. Does that make sense? If so, I wonder what information can we give them to provide in the support ticket so that it streamlines the debugging process.

Also @choumx and @samitron7 for thoughts on this.

[samitron7]

This suggestion looks sound to me @o-fernandez. For filing support ticket, we should look into adding a feedback button with dropdown options inside the editor as a northstar. But plus one to adding a dropdown and pre-populating issues when possible in our feedback forum.

[choumx]

That sounds fine. I think a single check with an exponential backoff retry should suffice. We should make sure to send a GA event when these fail to track volume.