Bedrock: Create download button variant for A/B experiment.

[amychurchwell]

Description

Add a download icon and accompanying hover state animation to the “Download Firefox” button to A/B test against the current download button.

Test Plan

https://docs.google.com/document/d/1f-NZmp2ROtPcuwFqcsq8X83Opj058STwCOcKKV5PLfY/edit

Tasks

• [x] Put proposed button on demo for design review.
• [x] Code experiment for final review.

[amychurchwell]

@dzingeek hi Jennifer! You mentioned the other day that you possibly have an asset for this?
Looking for a download icon- preferably a down-facing arrow. ⬇️

[stephaniehobson]

Download icon: https://github.com/mozilla/protocol-assets/blob/master/icons/download.svg

[amychurchwell]

@cgonz0 & @justinkropp
here is a demo link with current draft of button variations:
https://www-demo-demo-achurchwe-mbhddu.herokuapp.com/en-US/firefox/new/

[hoosteeno]

The design team provided feedback yesterday; next steps are coding for final review.

[alexgibson]

@amychurchwell Some feedback on the draft experiment in Convert:

As a general guideline, please try and follow the documentation for QA'ing Convert experiments. The process is to draft your experiment against staging, which I can then activate and review.

I took a look at the config and here is some initial feedback:

• The targeting URL should point to an experimental /exp/ page, and be configured against staging for QA: https://www.allizom.org/en-US/exp/firefox/new/
• The current goal is "Decrease Bounce Rate". You may want to change this to "Click: Desktop Download button on /new"
• On the animated icon variation, arrow icon seems off-center with the line underneath it? (see image below). The animation also feels a little on the slow side, but perhaps that's just me.

[amychurchwell]

@alexgibson Thanks for the feedback!

• Do you have time this week for a quick chat about convert because I was thinking I had configured it against staging (allizom)?
• Will update goal.
• Animated version is on the slower side, this was reviewed by Cristina & Kropp. We had done a couple different speeds & ended up landing on this one. I will double check the alignment, things had shifted slightly when I moved the code into the tool.

[alexgibson]

This is what I see in the targeting area:

Happy to walk through things if it may help, feel free to send my a zoom invite for any time that works 👍

[amychurchwell]

@alexgibson oh, I see! no, this clears things up. Thank you. 👍

[stephaniehobson]

• [x] The experiment’s logic
• ~Any JS included (in Convert editor’s JS field)~ n/a
• [x] Any CSS included (in Convert editor’s CSS field)
• [x] The target audience configured
• [x] The goals configured
• [x] The distribution percentages configured
  
  
  
  • I increased this to 100%, we will configure the 8% distribution on the CDN when the experiment is activated.
  
  
• [x] The target URLs configured

R+ ⬇️

I won't block the experiment on this but something to note if we want to make it permanent: because the CSS for positioning the background image is relative to the left side of the container, text that is wider than expected overlaps it. Not a problem for this experiment because it's en-US only and the text width does not change substantially if the font doesn't load.

[alexgibson]

This experiment is now running in production for 8% of traffic to /en-US/firefox/new/

[alexgibson]

This experiment has now been running for 11 days, and as had over 100,000 visitors (over twice the minimum sample size set out in the plan). The report isn't showing any statistical signifacance for either variation.

Can this experiment be shut off?

[amychurchwell]

@alexgibson it looks to have been paused on my end.