Wp-calypso: Domain Verification: Remove the nudge from the sidebar and add to checklist

[apeatling]

Once a user has completed signup they see the verification nudge in the sidebar:

In all user tests so far we've seen users distracted by this nudge and take themselves out of the website building flow and lost in the domain verification flow. Verification isn't important until they are ready to launch their site.

Let's start by removing the "Please verify [domain name]" from the sidebar, and replacing it with a checklist item. It should come as the step before launching your homepage:

[michaeldcain]

@apeatling @kelliepeterson @rickybanister @ianstewart: would it make sense to hide the nudge on the initial customer home (checklist) screen, but show it on others?

This would make sure that the users doesn't get distracted by it when they first land in Calypso, yet also don't lose track of it over time. Checklist items are non-blocking for launch and we don't know how frequently users return to the Customer Home screen.

[apeatling]

We have the "My Home" sidebar item now, does this make it less likely to miss?

I think we could try just removing it here for the next test, and see if the user still gets distracted at a later point.

What if this step was blocking for launch? Although the current verification as a sidebar nudge also doesn't block launch.

[rralian]

I agree with shifting the verification nudge to the checklist during the signup flow, but I'm not as sure about a change across all sessions without understanding Home usage and how this impacts domain verification. Would we run this as a test? We should coordinate with @Automattic/cobalt to make sure we're watching relevant numbers. Maybe it makes sense as a more prominent sidebar nudge at a certain age.

Making this a blocking step for launch sounds promising. Though users can own domains that are not attached to an active site (partially refund the plan, or purchase via wordpress.com/domains). So it may not be adequate on its own.

As for the "second domain" prompt that seems like it'd make sense to hide during the initial session (or maybe the first day?) and then show on a subsequent visit. @kelliepeterson what do you think?

[rickybanister]

That all sounds right @rralian. We're happy to test to start, but I feel pretty strongly that we needn't show these things to customers on their first run through.

[adamziel]

PR available at https://github.com/Automattic/wp-calypso/pull/37510

I agree about the A/B test approach though - we should update that PR to reflect that.

[kelliepeterson]

Sorry folks - didn't have email pings turned on. Fixed that.

I think most of this sounds super reasonable. One thing I do want to point out - registrants only have 15 days to verify their domain or it is suspended. That drives customers to Happiness, so I would like to make sure that whatever is done doesn't wind up a) driving added HE interactions and b) wind up bumming customers out because they didn't know that they had to take that action by a certain time.

Of course, users still get the email so I'm hopeful they act regardless but something to keep in mind.

For the Second Domain nudge I think we can suppress it for the first week or more even without it likely impacting performance.

[rralian]

We're happy to test to start, but I feel pretty strongly that we needn't show these things to customers on their first run through.

Totally agree on the outcome. I just worry about making changes without knowing the impact on other metrics, even if it's a 90/10 test. But I get that we don't want tests to slow us down from addressing what seem like clear UX issues. So I defer to the experts @kelliepeterson and @Automattic/cobalt on the necessity of abtesting. :-)

[klimeryk]

Added my take on the issue on the PR: https://github.com/Automattic/wp-calypso/pull/37510#issuecomment-553624510

[klimeryk]

This issue is actually about two, unrelated notices:

• verification nudge
• upsell for a second domain

I've split out the second one into a separate issue: https://github.com/Automattic/wp-calypso/issues/37652 and provided a fix for it here: https://github.com/Automattic/wp-calypso/pull/37653