Wp-calypso: [EPIC] Add tracking to events to Gutenboarding

[ramonjd]

This task needs to be further defined, but the gist of it is that we'll need to add as many events as we can in order to compare this flow with old start flow.

The advice in pbAok1-Ao-p2#comment-1260 should guide us as we go.

Tasks

• [x] Audit/create a diagram for the Gutenberg flow to get a visual aid for tracks flow events. See: https://github.com/Automattic/wp-calypso/issues/41059#issuecomment-613765368
• [x] Audit the current sign up flow for analytics events
• [x] ~Decide, or ask whether we should use existing signup names, e.g., calypos_{...} or a new one, e.g., gutenboarding_{...}~ ~We'll use current IDs, with with different flow values See: pbAok1-Ao-p2#comment-1260~ We're now using calypso_newsite_* https://github.com/Automattic/wp-calypso/issues/41838

Adding events for Gutenboarding

• [x] Implement starting and completing signup events, try to reuse calypso_signup_start and calypso_signup_complete https://github.com/Automattic/wp-calypso/pull/41129
• [x] Implement start and end events for each step, e.g., intent gathering, design and so on. #41167
• [x] Add properties to step tracking https://github.com/Automattic/wp-calypso/pull/41431
• [x] Implement open and close events for modals, e.g., domain picker. See: pbAok1-Ao-p2#comment-1267
• [x] ~Implement button_click events on Next/Back buttons https://github.com/Automattic/wp-calypso/pull/41462~ Looks like we don't need this for track right now pbAok1-Ao-p2#comment-1505
• [x] Implement a generic event for errors like calypso_signup_error #41273
• [x] ~Where we've mixed the flow with existing pages, e.g., plans or login, ensure that we're passing the correct flow name (assuming we have to). For the plans page,~ the signup framework uses the name of the flow, which is prelaunch We're already tracking these steps in the original signup code
• [x] Add traintracks tracking for domains Example #41173

Future iteration tasks

• [ ] Add Google analytics to Gutenboarding, and use it for click-based events. Probably a separate task/mini-project given the scope of the tracks migration into /packages

Registering events

⚠️ For each new tracking even we create, we'll need to "register them" See: pbAok1-Ao-p2#comment-1260

Events registered so far:

• calypso_signup_leave https://github.com/Automattic/tracks-events-registration/pull/241 (introduced by https://github.com/Automattic/wp-calypso/pull/41167)
• calypso_signup_error https://github.com/Automattic/tracks-events-registration/pull/240 (introduced by https://github.com/Automattic/wp-calypso/pull/41273)

[ramonjd]

Hi @roo2 ! Do you mind tackling this one next up?

A good place to start would be to see what we're tracking on the current signup flow, and find the most appropriate parallel in Gutenboarding.

As for the event IDs, we could use the same ID and track ID pattern for Gutenboarding events as we have in the current signup at least for development PRs. We can always find and replace them after we get advice from the data team.

Thanks!! 🙇

[roo2]

Yep l'll look at adding the events suggested by martin now!

[ramonjd]

I've created a very rough diagram of the flow/steps.

Updated to show that the plans steps (and probably the checkout steps) already fire events we can use in any funnels that begin in the Gutenboarding flow.

[ramonjd]

Quoting from pbAok1-Ao-p2#comment-1260:

For your tracking, you should be able to have just a couple of events:

1. calypso_signup_step_enter
2. calypso_signup_step_leave

Where every event has these two properties:

1. flow with a value like gutenboarding or gb202004
2. variant:domain_first (I don’t know if that specific value makes sense, but if you’re testing different orderings, you could use a variant prop like this OR you could keep a list of variants on a P2 and use names like gb202004_a or something.
3. step for the current step name, e.g. ‘domain’.
4. Optional: One cool thing to include on every _leave event would be a property for time_in_stepto record how many seconds they spent in that step.

So for example intent_gathering might be one step:

someTrackingCall( 'calypso_signup_step_enter', {
    flow: 'gutenboarding',
    step: 'intent_gathering',
    variant: 'some_variant_id`, // something referring to "soft launch" for example?
} );

// Let's leave!

someTrackingCall( 'calypso_signup_step_leave', {
    flow: 'gutenboarding',
    step: 'intent_gathering',
    vertical_id: 'some_vertical_id',
    site_title: 'lol_wut',
    variant: 'some_variant_id`, // something referring to "soft launch" for example?
    time_in_step: 123123 // this might be good for a future iteration?
} );

[simison]

@ramonjd you can find quite extensive flow diagram also from Figma file.

[ramonjd]

you can find quite extensive flow diagram also from Figma file.

Thanks. I basically copied it from there so I wouldn't have to open Figma again for at least another two days :trollface:

[simison]

I don't want the use of "gutenboarding" term to move focus to just the initial few steps of onboarding; let's also track what goes on in later steps. Namely, I'd like us to track the "launch" button and "welcome modal" in wp-admin.

[ramonjd]

For the plans page, the signup framework uses the name of the flow, which is prelaunch

We could make it consistent with whatever flow id we're using for Gutenboarding (gutenboarding, new, gb_signup or whatever), but it's unique enough I think.

And so long as we get the start and complete events right in our flow, we can add everything in between to funnels, even plans/checkout events that already exist.

I've updated the WIP diagram to reflect this.

[ramonjd]

let's also track what goes on in later steps. Namely, I'd like us to track the "launch" button and "welcome modal" in wp-admin.

What do you mean by "wp-admin" in this context? Is there some point at which the Gutenberg flow takes us into wp-admin?

Tracking the launch button

I _think_ it should be straight forward to add an event listener to the launch button, seeing as we build it in gutenboarding overrides, and we're using the tracks library in /apps it seems anyway.

However I don't even think we need to right now.

Once user clicks on the launch button from the editor, they will land on ${ window.location.origin }/start/frankenflow?siteSlug=${ this.props.siteId }.

The launch "flow" beyond that click is still called 'frankenflow' (which I guess we should change at some point too). Since it's part of the signup framework, every step in the flow is tracked already, through the calypso_signup_step_start and calypso_signup_actions_complete_step events among others.

By the way, the name of the flow isn't really important to build funnels, so long as we know what it is and it's unique. 'frankenflow' is certainly unique enough 🤣 but we might like to change it to 'new-launch' or something.

Tracking the welcome modal

As for the welcome modal, we mean this one (?):

If what I'm seeing is correct, and it's part of the FSE plugin, I'd tend to think that it's beyond the scope of this specific round of analytics.

FSE has implemented a small tracking library for the page template modal, which we could probably copy or extend. We might have to get in touch with @marekhrabe to make sure.

What do we want to track exactly on the welcome modal? Clicks? Dismissals?

[ramonjd]

Some inspiration for whenever we implement a time_in_step property

p7H4VZ-2x2-p2

[ramonjd]

The groundwork for tracking is done for Gutenboarding. The next tasks will be refining, testing in funnels.