Polaris-react: [List filters] Design merchant opt-in/out flow

[sarahill]

Problem

The List Filters update will affect a lot of different list views throughout the Shopify Admin. We don't currently have an effective way to allow merchants to opt-in or turn on a feature on their own time. Automatically turning on features that could potentially slow down a merchants workflow can be frustrating.

Solution

Allow the merchant to choose when they want to turn on an early access feature. The idea behind Early access is to be upfront with merchants that something within a heavily trafficked part of the UI will soon be changing. This is their oportunity to turn on that exerience early, try it out, and send us feedback on how we can improve the feature. During the Early access period, the merchant will be able to Switch back to the previous experience at anytime.

Get early access card

The early access calls out the new feature, briefly explains the benefits, and gives the merchant a solid idea of when to expect the feature to be fully released.

Actions

• Get early access enables the new filters
• Update in a month Dismiss the ability to opt in early to the beta. These merchants will get the new filters with the 100% rollout.

Rationale

Get early access reiterates that the merchant is opting into a feature before it's been officially released. It also ties in with the Early access badge.

We have replaced Remind me later and Dismiss with Update in a month. Since we have a plan to release to 100% shortly after the beta release, we believe giving the options to either opt in early or update later add more clarity to what Early access means.

---

Fullstory acceptance

Using Fullstory tracking will give us the opportunity to see the merchant's first time experience using the new feature. However, we are cognizant of the fact that Fullstory can be seen as invasive and is essentially taking a video of the merchant's usage inside of the Shopify admin.

The acceptance steps gives full transparency to the merchant and allows them to Allow sharing or Skip this part of the early access.

• Allow sharing Merchant consent for Fullstory tracking
• Skip Merchant declines Fullstory tracking
• Switch back to old filters Reverts back to the old filtering design. The early access card will display, giving the merchant the ability to turn on the new filters again if they want.

---

Design and content

• Design
• Content spec

[sarahill]

Submitted an illustration request today, 2/15/2019, for the opt-in banner. I think we can ship without it if the timing does not work but I believe it would help call attention and help promote the feature.

[sarahill]

A couple of explorations for the opt in banner:

This is using the callout card structure

This is pulled from the new feature indicator banner

[alekmackie]

SVG of this illustration attached:

list-filters-callout.svg.zip

Sara, Selene and I worked through some opinions on this. We determined:

• The illustration and the badge compete for attention
• Since there isn't enough room for illustration to literally explain how this feature works, we need to rely on metaphor, which introduces considerable complexity, and makes the messaging more difficult to understand
• This isn't a typical callout card, so it doesn't _need_ an illustration, but it is important for this to have the Early access badge
• The illustration could be a good fit for messaging that's more strongly focused on the launch of a new feature over the ability to opt-in to this beta---like a more typical callout card used to communicate the widespread launch of this new feature (non-beta)

I'd be okay with this opt-in/out card to have the illustration, but I'd also be okay for it to be without :)

[sarahill]

@VanjaMitrovic and I paired on the opt in and fullstory acceptance flow.

Below is a proposed direction based on our exploration and discussion.

---

Proposed direction

TL;DR
Our goal with introducing Fullstory, is to gain insight into how a merchant interacts with the admin in real time. We want to know their first impressions. Fullstory can help us cast a wider net without purely relying on 1:1 merchant testing. However, not everyone is comfortable with tools like Fullstory and we don't want this to be a barrier to entry for the opt in beta.

We're reccomending breaking up opt-in and acceptance into two separate steps. Breaking these steps up adds clarity to what we're trying to convey at each point. We're introducing this info as way to make merchants aware of what they’re opting into and what Shopify’s intent is.

Opt in

Messaging around new list filters is displayed in a calloutCard with information the feature itself and options to [Try the new filters] [Remind me later] and [Dismiss].

• [Try the new filters] this will display the new filters in the UI and prompt the Fullstory acceptance modal to display.
• [Remind me later] dismisses the opt-in banner for a period of time.
• [Dismiss] breaks from the usual x pattern for closing a card, banner, or modal dialogue. In this context we want to be clear that x isn't just closing this card but Dismissing the ability to opt into the feature.

Acceptance

The acceptance message for Fullstory displays in a modal after the merchant has clicked [Try the new filters].

[sarahill]

Based on feedback from the UX Critique with Kyle P. yesterday:

Design and content to explore a couple options around the Opt-in + Fullstory acceptance:

• Showing the acceptance modal at some point after the merchant has already opted in
• Make the acceptance modal more meaning. Onboarding step + acceptance

Current direction and flow: https://shopify.invisionapp.com/share/VRQVCGK8PA9

[sarahill]

@VanjaMitrovic @selenehinkley

content for opt in banner: should we be explicit about what early access means? We talked about being explicit with the opt in message: the filters will be released in n number of days/weeks but you can turn the update on early.

Example for GMail

[sarahill]

Ryan, Vanja, Sonja, and I paired on Friday, March 8, 2019, to flesh out the designs for the Fullstory acceptance and Send feedback on opt out.

---

Fullstory acceptance

For Fullstory acceptance, we are proposing displaying the message in the card after the merchant has opted-in. There will be three options along with the Fullstory acceptance text: Allow, Skip, Switch back.

Rationale:
We ultimately felt moving away from the modal was the best direction in this context because, as Kyle stated in the last critique, this is benefiting Shopify more than the merchant.

Try the new filters

Fullstory acceptance

Trying the new filters

---

Feedback when opting out

Feedback flow for switch back: When the merchant decides to Switch back we decided to display a toast with a confirmation and a link to Send feedback.

Rationale:
We recognize that we will most likely receive fewer pieces of feedback but, again, this is benefiting Shopify more than the merchant. The feedback will also more likely be of higher quality since it will be more of a deliberate action.

Switch back, send feedback

---

Designs can be seen here:
https://shopify.invisionapp.com/share/DTQX7PIKPGF

[StephPoce]

@ry5n @sarahill @VanjaMitrovic @yourpalsonja

The removal of the modal is great, it makes the transition from the old to new experience very smooth.

In regards to the Fullstory acceptance:

• There are two types of actions in the card but the content is very focused around the tracking acceptance. This may create confusion around the switch back mechanism. Is there a way to more clearly articulate what the 'switch back' link will do?

• The card actions are very directive. From the merchants perspective do you think that they will feel that they have to take action in this card? Are the actions clear enough that the merchant will understand what happens next? (Decline seems unclear to me and as mentioned above switch back lacks context. The design of 'Allow sharing' as a button focuses attention).

• The original banner with the line 'showing the new way to filter orders' offered a nice lead in to the experience to help orient the merchant and help provide feedback to the merchant after they opted-in. Is there a rationale for removing it from this card? Was it a space issue?

Opting out toast

The rationale makes a lot of sense and I think the implementation works as it removes the opt-out feedback form as a blocking step.

[VanjaMitrovic]

since we're limited to a couple of lines for the banners, I wanted to make sure that the most important information is mentioned.

Proposed opt-in/out content:
Get early access to more control over filters (Early access)

In less than 3 weeks, the way to filter orders will be updated. You’ll be able to rename custom tabs and combine more than one filter from the same category.

[Get early access] [Update in 3 weeks]

This is the content @selenehinkley and I worked on based on some feedback/ideas from Kyle and Sara. We felt it's important to include the benefits of the new filters to merchants, so we included the top two (?) requested features.

[VanjaMitrovic]

From https://github.com/Shopify/shopify/issues/187810

Proposed FullStory opt-in/out content for the list filters project:

Improve filters by sharing your activity with us

We'll temporarily track what you click on in Shopify to measure how it's working for you. View our [privacy policy].

[Allow sharing] [Skip] [Switch back to old filters]

We're limited to a couple of lines in the banner, so we tried to include the most important points. we re-framed the content to focus on the merchant benefit, not Shopify's, based on Kyle's feedback.
do we think this is enough/the right information to convey?

[ry5n]

For the early access content:

• Since we have the badge saying “Early access”, can we simplify the title? “Get more control over filters”
• Can we be a little looser about the timeline, just in case? “In about 3 weeks…”
• Nit: in the second body sentence, “combine” might be harder to understand than a simpler verb.
• Can you explain the rationale for the change in the actions? Why change from [Try the new filters]?

I feel good about the Fullstory content.

[dpersing]

Some notes on focus management:

• When a merchant activates Try the new list filters, is there a page refresh, or does the list filter section and banner update automatically?
• In general, any time the content updates dynamically we'll need to make sure focus isn't lost when the content inside the banner changes. Focus should get moves to the next version of the banner when it updates.

[VanjaMitrovic]

• Sure, we can get rid of that. Good point
• About or around might work.
• We struggled with this message, especially finding a verb that is both simpler and more accurate to describe the action. any ideas come to mind @ry5n?
• The rationale behind changing the CTA to [Get early access] is that it implies that this change is going to be "permanent" in a few weeks time. It also ties the CTA to the heading with the early access badge, making it more consistent/clear.

[sarahill]

[In reference to Fullstory acceptance]

The original banner with the line 'showing the new way to filter orders' offered a nice lead in to the experience to help orient the merchant and help provide feedback to the merchant after they opted-in. Is there a rationale for removing it from this card? Was it a space issue?

☝️ I believe our thought with removing a title from the acceptance step was to focus the merchant on the actionable content. @VanjaMitrovic I just noticed you did add a title to the newest content updates: Improve filters by sharing your activity with us.

So it would look something like:

[ry5n]

^ I’m not a big fan of the Fullstory opt-in (second step) having a title, I don’t think it merits that level of emphasis

[sarahill]

Yea, it feels empty without a title but with a title it start to feel too much like the opt in banner. Will merchants be confused by this?

[VanjaMitrovic]

I think if we remove the banner the concern was that they don't read it at all, which seems fine. we also don't want it to feel compulsory. let's remove the header and add it to the body and see how that looks. also, we should add a period after privacy policy.

[dpersing]

Oh! Re the privacy policy link. I put a comment on the Invision file but it was a few versions ago. Does that link open a new tab/window? We'll just want to be aware of https://github.com/Shopify/polaris-ux/issues/146, since we don't have a consistent way of conveying that change of context.

[tmlayton]

is there a page refresh

Sort of, we need to trigger a query refetch for the shop data which will then update the DOM and show a toast message. The banner does not update, it is replaced. Although maybe we could build this out as a single component with a shared container.

[sarahill]

Opt in card

Vanja updated the content for the opt in card to be clear about this being early access.

Actions:
Get early access enables the new filters
Update in a month Dismiss the ability to opt in early to the beta. These merchants will get the new filters with the 100% rollout.

Rationale:
Get early access reiterates that the merchant is opting into a feature before it's been officially released. It also ties in with the Early access badge.

We have replaced Remind me later and Dismiss with Update in a month. Since we have a plan to release to 100% shortly after the beta release, we believe giving the options to either opt in early or update later add more clarity to what Early access means.

---

Fullstory acceptance

Vanja worked out the content for the opt in banner and full story acceptance. The actions in the fullstory acceptance card are now more clear: Allow sharing Skip Switch back to old filters

---

Latest designs:
https://shopify.invisionapp.com/share/TYQYTIXZV6R

[ry5n]

I’m happy with this flow, and the copy. Steph sent the latest flow in Invision ^ to Kyle for feedback.

[sarahill]

@StephPoce and @selenehinkley updated the content for Fullstory acceptance based on feedback from Lynsey Thornton, Kyle, and Yesenia.

---

Fullstory acceptance

Goals of this content:

We want to:

• let merchants know they have the option to help us improve filters by allowing us to temporarily track their activity
• clarify that "activity" includes their interactions with filters in Shopify
• let merchants who care about privacy details to read more in our privacy policy (extra trust factor)

Tone goals: conversational, encouraging, educational, and most importantly, transparent

Pro tip: put yourselves in a merchant's shoes. We cater to merchants who read at a grade 7-9 reading level and who don't have a university education. This is why we refrain from UX terms like:

• Experience
• Feature
• Data usage. Data usage is also a politically charged term and doesn't transparently represent what fullstory does.

What's new:

• We're using "help us understand" language commonly used by researchers (as recommended by Lynsey)
• We clarified what "activity" means by specifying it's "Shopify screen activity", and broken down, that means "how you interact with filters"
• We made the sentences more conversational and added more "we" and "our" language (something we normally refrain from in the admin, but we decided to make Shopify more of a focal point of the conversation to increase trust and add the "help us" factor for those merchants that feel good from helping out).
-

Help us understand how list filters are working by sharing your Shopify screen activity with us. We'll temporarily track how you interact with filters so we can learn how they're used. For more information, view our privacy policy

[Allow sharing] [Skip]