Foundation.mozilla.org: PNI Usertesting of new designs

[beccaklam]

placeholder ticket for usertesting of new PNI designs

[taisdesouzalessa]

Script is in progress but first need to collect Becca's input on what we should test vs what can be left out of the testing.

WIP script + thinking is here

[taisdesouzalessa]

Script (pages 1 to 4) is here

Next steps:

• [x] collect Becca's feedback
• [x] transfer test to Usertesting.com
• [x] do a QA of the test with somebody who is not involved on the project (Becca or I) to increase its quality
• [x] apply changes to Usertesting.com
• [x] launch test

@beccaklam

[kristinashu]

I've opened a ticket for reviewing and synthesizing the results https://github.com/mozilla/foundation.mozilla.org/issues/4013

How many test are you planning to do?

[taisdesouzalessa]

Thanks for opening that ticket, Kristina.

The plan is to do 2 test with 10 users:

• 5 desktop users: sticky creepometer variant
• 5 desktop users: button creepometer variant

UPDATE: we decided to drop off the mobile test because the things we want to test are either device-independent or desktop related (sticky bar vs button layout). Since we have to meet the quota of 2 tests/month with MoCo we opted for dropping the mobile test:

• 5 mobile users: there is only one variant so we will be testing that.

cc: @kristinashu

[kristinashu]

Ah ok thanks, sorry I see your comment in the doc now. So 15 users x 15 min = 3:45 hours to review sounds reasonable.

[beccaklam]

Invision prototypes are all ready. The file is here. I've shared it with @taisdesouzalessa and @kristinashu and Nat, just in case.
I've also merged all my testing designs into abstract. You can find it in the PNI file and looking for the tab "PNI Test 2019".

Tais has written and uploaded the user tests to usertesting.com you can find them here:
Variant A (Side Bar)
Variant B (Sticky Button)

Currently just waiting on feedback from Jen.

[beccaklam]

Oh, some more assets in case you need them: https://drive.google.com/drive/folders/1g_UYJ161dnTbev-HONisdJ_E55DIs3tX?usp=sharing cc: @taisdesouzalessa @kristinashu

[kristinashu]

Is there a difference in these two home screens?

I noticed the About page in the nav looked selected so I updated one but then was worried there's something subtly different about the two that I'm not aware of.

[taisdesouzalessa]

Test has been launched:

• 5 users will see Variant A (sticky voting window)
• 5 users will see Variant B (voting button)

Test script is here

User Testing platform:
Privacy Not Included - variant A
Privacy Not Included - variant B

NOTE from MoCo: don’t forget to give star ratings to participants, it helps them, and if you get a really terrible one or a session that doesn’t work, you can let an admin know and we can request a new one.

@kristinashu @beccaklam

[beccaklam]

Is there a difference in these two home screens?

I noticed the About page in the nav looked selected so I updated one but then was worried there's something subtly different about the two that I'm not aware of.

Hmm, is that in Sketch? If yes, there shouldn't be a difference. In InVision there are two different Home screens because they link to different product layouts.

[kristinashu]

Oh it was in InVision but test are done now! Gonna close this ticket!

Results will be here https://github.com/mozilla/foundation.mozilla.org/issues/4013.