Wp-calypso: Gutenboarding: make sure we use domains API correctly to get relevant results

[simison]

[...] my site was about photography and none of the top 5 sites are .com related and all are .blog related. not sure how we can have better visibility into these rules and logic.

via p1580154057008900-slack-mp-domain-step-optimization

Turns out, we are using the domains APIs a little bit incorrectly.

We use:

https://public-api.wordpress.com/rest/v1.1/domains/suggestions?include_wordpressdotcom=true&include_dotblogsubdomain=true&only_wordpressdotcom=false&quantity=1&vendor=variation2_front&vertical=p31v28&query=photo%20studio&_locale=user

For fully qualified domains we want to do:

https://public-api.wordpress.com/rest/v1.1/domains/suggestions?include_wordpressdotcom=true&quantity=10&vertical=p31v28&query=photo%20studio

If we want both subdomains and FQDN, we’ll want to do two separate queries.

Above API query explanation from p1580156280019600-slack-mp-domain-step-optimization

Documentation: p1586443547470200-slack-cobalt

[yansern]

Just to make things easier to look at.

First API Request

The first API request returns 10 domain suggestion, 1 free subdomain from *.wordpress.com and 9 paid domains.

Request:

https://public-api.wordpress.com/rest/v1.1/domains/suggestions

• include_wordpressdotcom: true
• quantity: 10
• vertical: p31v28
• query: photo studio

Response:

[
  {
    "domain_name": "****.wordpress.com",
    "cost": "Free",
    "is_free": true
  },
  {
    "domain_name": "studio.photo",
    "relevance": 1,
    "supports_privacy": true,
    "vendor": "donuts",
    "match_reasons": [
      "tld-exact"
    ],
    "cost": "$25.00",
    "product_id": 351,
    "product_slug": "dotphoto_domain"
  },
  // ... and 8 more paid domains
]

Second API Request

The second API request from the example above returns free .blog subdomains:

Request:

https://public-api.wordpress.com/rest/v1.1/domains/suggestions?

• include_wordpressdotcom: true
• include_dotblogsubdomain: true
• only_wordpressdotcom: false
• quantity: 1
• vendor: variation2_front
• vertical: p31v28
• query: photo studio

Response:

[
  {
    "domain_name": "photostudio.design.blog",
    "cost": "Free",
    "is_free": true
  }
]

Suggestion

UPDATE: No longer relevant.

The params for API request to return purely paid domains should look like this:

- vertical =  p31v28
- query =  photo studio
- quantity = 10

The params for API request to return purely free subdomains (both *.wordpress.com subdomain and *.your_category_here.blog* subdomains):

- include_wordpressdotcom: true
- include_dotblogsubdomain: true
- only_wordpressdotcom: false
- quantity: 10
- vertical: p31v28
- query: photo studio
- vendor: variation2_front

I'm not sure what the vendor param is for though.

[klimeryk]

The params for API request to return purely free subdomains (both .wordpress.com subdomain and *.your_category_here.blog subdomains):

That's not exactly correct - that's only true if the user selected the Blog segment/site type. And since that's not a thing anymore, we should not suggest .blog subdomains anymore, AFAIK.
Also, no need to request 10 free subdomain suggestions - one should be sufficient.

Overall, I'd simply suggest inspecting what current onboarding does and mimic that :) And the above parameters are not in sync with that.
vendor for free domains does not matter that much, but I'd love it if you used the same vendor value as Calypso, not a hardcoded one: https://github.com/Automattic/wp-calypso/blob/e1face4840c4d7710a66d76d90bf1ff810dede8b/client/lib/domains/suggestions/index.js#L7 (is Gutenboarding signup or not? ;))

TL;DR: happy to discuss this further on Slack or P2 :)

[sirreal]

Related - #40236 is adding a hook for the free domain: https://github.com/Automattic/wp-calypso/pull/40236/files#diff-1079e690e56573527885526f8fd10cd5

[yansern]

@klimeryk Thanks for the clarifications! I understand it now!

And since that's not a thing anymore, we should not suggest .blog subdomains anymore, AFAIK.

If that is the case, there is simply no need to create a secondary API request to retrieve .blog subdomains anymore. we'll just need to set the param include_dotblogsubdomain to false.

Overall, I'd simply suggest inspecting what current onboarding does and mimic that :) And the above parameters are not in sync with that.

Will do so! I am now awaiting my punishment for deviating from the specs!

Is Gutenboarding signup or not? ;))

My common sense says it is signup, but the sample API request provided uses variation2_front, which is not signup. I'm just going to stick with variation2_front.

@klimeryk What's the difference in the results that is returned using variation4_front and variation2_front?

And thanks for pointing me to wp-calypso/client/lib/domains/suggestions/index.js.

[yansern]

Related - #40236 is adding a hook for the free domain: https://github.com/Automattic/wp-calypso/pull/40236/files#diff-1079e690e56573527885526f8fd10cd5

@sirreal Thanks for the heads up! I don't know how you keep track of all these new developments!

[yansern]

@simison

If we want both subdomains and FQDN, we’ll want to do two separate queries.

• We want only one *.wordpress.com subdomain.
• We don't want .blog subdomains. (See https://github.com/Automattic/wp-calypso/issues/40164#issuecomment-600292227 on .blog subdomains should not be suggested anymore.)

The solution is to set include_dotblogsubdomain: false on the existing API request. And that will be only 1 query needed. That's all!

The two files that needs this line added are:

• header/index.tsx
• domain-picker-button/index.tsx

Once this PR https://github.com/Automattic/wp-calypso/pull/40236 which refactors freeDomainSuggestion into a hook is merged in, I'll add the relevant changes in another PR.

[klimeryk]

The solution is to set include_dotblogsubdomain: false on the existing API request. And that will be only 1 query needed. That's all!

The reason it was/is split in the current onboarding flow is because the WPCOM suggestions were much slower than our custom domain suggestions. In the meantime, we've worked on improving the performance (usually at the cost of accuracy), so it might indeed work for you. But just wanted to mention the reasoning for it :) For short, very generic/popular queries, there might still be a bit of a slow down, but nothing like it used to be.

Please also be aware of https://github.com/Automattic/wp-calypso/issues/40198.

@klimeryk What's the difference in the results that is returned using variation4_front and variation2_front?

The thing is, you should not worry about that and use the value provided by the getSuggestionsVendor function. You should definitely _not_ hardcode it to either variation4_front or variation2_front. These values were often A/B tested in the past and could be again in the future. And I'd love it if the vendor was consistently chosen based on a central source of truth.

The names are vague for a reason - to avoid our vendors knowing which results are coming from which vendor ;) And to keep the A/B test more "anonymous".

Will do so! I am now awaiting my punishment for deviating from the specs!

Hahah, no worries - it's just that the suggestion endpoint has been modified many times, with parameters being added, etc. So at this point it's quite brittle and the safest bet is to use what existing onboarding uses. And even that does not guarantee proper results, as https://github.com/Automattic/wp-calypso/issues/40198 shows... ;)

[yansern]

@klimeryk Thanks for taking the time & effort to explain the context and the reasoning behind things! Much appreciated!

[simison]

@klimeryk just wanted to double-check that doing one API call with these defaults should yield good results?

https://github.com/Automattic/wp-calypso/blob/58442f38b9682f90c2822458b213e552155a42d3/packages/data-stores/src/domain-suggestions/selectors.ts#L75-L86

The thing is, you should not worry about that and use the value provided by the getSuggestionsVendor function. You should definitely not hardcode it to either variation4_front or variation2_front.

I don't see getSuggestionsVendor in packages/data-stores/src/domain-suggestions currently used. Is that an issue?

[klimeryk]

@klimeryk just wanted to double-check that doing one API call with these defaults should yield good results?

That depends on what are your expectations? Should there always be one free WPCOM subdomain and 4 paid suggestions? Or a mix? Is your code handling cases where there's either no free or paid suggestions? I'd have to double-check the backend code, but all of that seems possible, so "good results" can be quite subjective ;)

include_wordpressdotcom: queryOptions.only_wordpressdotcom < this line looks quite confusing, especially when in the actual query only_wordpressdotcom is hardcoded to false 😬

I don't see getSuggestionsVendor in packages/data-stores/src/domain-suggestions currently used. Is that an issue?

Unfortunately, yes, it is a possible issue - in the past, we've had to switch providers for many reasons (a/b testing being the usual one, but also during outages or other problems with providers). In order to provide consistent user experience, we need to ensure we use the same provider across the board. It will probably seriously screw the results of an A/B test as well ;) We don't have short-term plans of testing any new providers, but nevertheless we should make sure the same provider is used consistently.

[simison]

That depends on what are your expectations? Should there always be one free WPCOM subdomain and 4 paid suggestions? Or a mix? Is your code handling cases where there's either no free or paid suggestions?

Yes, there should _always_ (no matter what happens) be one free domain available, and a bunch of paid options. Can that happen with search terms? I think I saw a cap on randomized rounds of free domains at API. 🤔 If so, should we improve the API or issue another API call for just the free domain?

getSuggestionsVendor

We'll get that sorted 👍

[simison]

@klimeryk looks like I linked you to the data layer where defaults were defined, and I should've linked this instead — it's where the query is actually built:

https://github.com/Automattic/wp-calypso/blob/53152de21e71deb004831778243cc855b1e9c372/client/landing/gutenboarding/hooks/use-domain-suggestions.ts#L29-L34

That depends on what are your expectations? Should there always be one free WPCOM subdomain and 4 paid suggestions? Or a mix? Is your code handling cases where there's either no free or paid suggestions? I'd have to double-check the backend code, but all of that seems possible, so "good results" can be quite subjective ;)

• one free domain (should _always_ get this no matter what)
• 4 paid domains
• one of paid domains "recommended"
• prices for paid domains

include_wordpressdotcom: queryOptions.only_wordpressdotcom < this line looks quite confusing, especially when in the actual query only_wordpressdotcom is hardcoded to false 😬

I now see there's a code comment in above-linked lines which might explain it: "Avoid only_wordpressdotcom — it seems to fail to find results sometimes "

getSuggestionsVendor

We'll get that sorted 👍

Tracking in https://github.com/Automattic/wp-calypso/issues/40968

As long as we change that, should that suffice as one API query to get what we'd need?

I see that in current domains page we are doing multiple queries to get results, but sometimes the free .wordpress.com query fails — we really need it always to succeed:

Might be that's why the query is with a combination of include_wordpressdotcom: true, include_dotblogsubdomain: false.

We talked about matching queries at our end with what's on Calypso's domain page. I wouldn't really "match what's in domains page" if we can risk getting no free results.

Thoughts?

[klimeryk]

Yeah, seems there's some regression or bug related to that. There's been a few changes in the recent times and the whole endpoint is a mess of different teams contributing to it to bend it to their will ;) There's been a bunch of changes in signup lately with whole steps removed and it's already been a brittle flow :/

I wonder if the simplest solution would simply be to create a new version of the endpoint that suits your needs? And then switch the signup to use it as well. Something that accepts something more reasonable and clearer:

wpcom_subdomains: 1
paid_suggestions: 5
vendor: variationx_front

I can help make that happen and it will be easier to debug than the current version of the endpoint. We were thinking of cleaning it up anyway, so it's a good nudge.

In the meantime I'll look into the issue with not being able to generate free WPCOM subdomain suggestions - that should not happen. Thanks for pointing it out!

Edit: looks like it was introduced with https://github.com/Automattic/wp-calypso/issues/40198

[simison]

Fresh endpoint sounds good to me. Does it even need the variation code at frontend, or can that be hidden to backend? API could receive signup=true flag if that's important for determining the vendor variant. Do you need the vertical info?

We also have a design of modal that opens when customer wants to see "more domains" and there different tlds are categorised to groups. I can share more info and designs, and our plans once I'm back from afk on Tuesday.

whole endpoint is a mess of different teams contributing to it to bend it to their will ;)

Happens. :-) It might make sense that you all just own the API and nobody else touches it. While we work on UI, I'd love to just issue one super simple query like you described.

And then switch the signup to use it as well.

— remember that this _is_ the new onboarding/signup flow if all goes well, and other flows will get deprecated and eventually removed.

[ockham]

Fresh endpoint sounds good to me. Does it even need the variation code at frontend, or can that be hidden to backend?

My thinking exactly :+1: It makes sense for a domain suggestions endpoint to expect arguments like a query, vertical, and maybe some flags about which TLDs (and/or subdomains) to include or exclude, but the vendor should IMO be considered an implementation detail. In other words, I can expect the user calling the endpoint to know what information to provide about the desired domain in order to provide meaningful search results, as all that's domain-specific. But the results provider/vendor? That should be treated as an implementation detail IMO -- how is a user calling the endpoint even supposed to know which vendors are available?

whole endpoint is a mess of different teams contributing to it to bend it to their will ;)

Sounds familiar :roll_eyes:

Happens. :-) It might make sense that you all just own the API and nobody else touches it.

Would it be possible to 'harden' any such endpoint against accidental breakage, e.g. by adding some unit(ish) tests that query a few domains for which we know some desired results "for a fact"?

[klimeryk]

It might make sense that you all just own the API and nobody else touches it. While we work on UI, I'd love to just issue one super simple query like you described.

We tried that but it does not work that way at a8c. Welcome to chaos :)

— remember that this is the new onboarding/signup flow if all goes well, and other flows will get deprecated and eventually removed.

Good point. Though, we've seen old versions of Calypso in the wild for weeks after that. Though, at some point of course we should be able to make the call to remove the old endpoint.

Does it even need the variation code at frontend, or can that be hidden to backend? API could receive signup=true flag if that's important for determining the vendor variant.

AFAIK, it's been done that way, because it makes A/B testing on the fronted easier. I'd prefer to preserve this. Unless you know of an easy way of passing the A/B test from backend to frontend.

That should be treated as an implementation detail IMO -- how is a user calling the endpoint even supposed to know which vendors are available?

I don't agree - depends how you look at it. The user here is Calypso and IMHO it's perfectly fine for it to select the vendor it needs depending on the context/flow. For example, what if Vendor 1 was best in English locale, but crap in others. And Vendor 2 was vice versa? We'd let Calypso choose based on the user's locale. It all depends where you want to put the control over it. It made sense for our needs to put it on the frontend, so that's where we put it.

Do you need the vertical info?

We'd love it - we recently started sending that to our domain suggestions provider to get more contextual results... and then promptly the vertical step was removed 🙄 If you don't supply it, we simply return more generic results. Sometimes that's good enough, but having the vertical definitely gives better results.

Would it be possible to 'harden' any such endpoint against accidental breakage, e.g. by adding some unit(ish) tests that query a few domains for which we know some desired results "for a fact"?

We already have tests around it (the quality of them is another thing). But since we're relying on a third party, results can change. demo can give x results today, y tomorrow. So depends what you mean by "desired" :)

[klimeryk]

Fix for the WPCOM subdomain bug in https://github.com/Automattic/wp-calypso/pull/40996 and D41668-code.

[razvanpapadopol]

getSuggestionsVendor

We'll get that sorted 👍

Tracking in https://github.com/Automattic/wp-calypso/issues/40968

Fixed in https://github.com/Automattic/wp-calypso/pull/41177