Jetpack: In-place Connection: Sometimes Approval redirects to the calypso site selector instead of a Plans page

[brbrr]

Sometimes during an In-Place connection user gets redirected to the Calypso site selector instead of a Plans page after connection approval. It happens a few times a day for E2E tests, but also some folks saw the same behavior in their manual tests.

The first failure was registered on October 17: p1602905919233600-slack-C1Y1F7ZM1

Steps to reproduce the issue

1. Start with a new site
2. Approve connection
3. See site selector

Screenshots

[rcanepa]

I have a hypothesis that could explain why sometimes we see this error.

Before explaining the hypothesis, I need to add some context that will be necessary to understand the problem.

Fact 1): by default, Calypso stores part of the app's state in a client-side cache, so by the second time a user visits wordpress.com, most of the data will be loaded directly from this cache. We include in this cache the list of sites associated with a user/account – this is key to understand what's going on because a recently created site won't be in this list right after a user connects it. It will be included only after the app receives a response from the /sites endpoint.

Fact 2): every time we visit a path that includes a site (e.g. wordpress.com/plans/:site), the siteSelector controller acts as a guard by checking whether the site in the URL is a valid site for the current user. It checks if the site is in the app's state (loaded from the cache), if the site is not found there, the app will send a request to /sites/:site in order to fetch its information. If for some reason, the endpoint returns something that resembles a Jetpack connection error (see code snippet below to see an example), the siteSelector controller will redirect the user to the SiteSelector component/page.

{
  "code": 400,
  "headers": [
    {
      "name": "Content-Type",
      "value": "application/json"
    }
  ],
  "body": {
    "error": "jetpack_error",
    "message": "The Jetpack site is inaccessible or returned an error: transport error - HTTP status code was not 200 (410) [-32300]"
  }
}

So, what's going on here? When we create a new site, that site won't be present in the client-side cache (for obvious reasons), so the siteSelection controller will request the site information to the /sites/:site endpoint. If this fails, for whatever reason, the user will be redirected to the SiteSelector component/page. So, this problem might affect any kind of site with an unstable connection.

This, in theory, should only happen the first time users visit wordpress.com right after a new site was connected. This is because when we visit wordpress.com, the app requests all users' sites to the /sites endpoint and the response will include the new site, so in the end, it will be added to the client-side cache.

We can more or less replicate a similar issue if we have a site that has a broken connection but it's present in our list of sites on WordPress.com. In the following capture, I have a site (ethnic-jay.jurassic.ninja) that has a broken connection but it's present in my list of sites. First, I delete the client-side cache (the list of sites is gone), and then I refresh the page. This forces the siteSelection controller to make a request to /sites/ethnic-jay.jurassic.ninja but since the connection is broken, no information about the site is returned. This makes the controller redirect me to the SiteSelector component/page.

If you're interested, the response I got from the endpoint was exactly the same I pasted above.

It would be great to have numbers to understand if this is a real issue for our users or not, and if so, for how many. I bet we can do something about this and improve the UX if necessary.

Lastly, some weeks ago I worked on a similar issue that was affecting Jetpack cloud. I was naive and tried to improve the siteSelection controller but, in the end, my proposal wasn't a good idea, but it served us to detect that we had the client-side cache disable on Jetpack cloud 🤦 .