Docs-website: signup_click event no longer occuring

[tariqahmed]

The signup_click event appears to no longer be firing/tracked.

The last occurrence of this was on March 4th.
SELECT * FROM PageAction where actionName ='signup_click' and appName ='docs.newrelic.com' since 2 weeks ago

OR For a time series view

SELECT count(*) as 'Sign Ups' FROM PageAction where actionName ='signup_click' and appName ='docs.newrelic.com' since 1 month ago timeseries 1 day COMPARE WITH 2 months ago limit max

Though this query does return event data

SELECT count(*) AS 'Signups' FROM TessenAction WHERE nr_product = 'DOC' AND eventName = 'TDOC_DocPageLinkClick_stitchedPathLinkClick' SINCE 1 day ago

[tariqahmed]

@jpvajda - FYI

[jpvajda]

We seems to be tracking the signup event in GA correctly, so I'm curious what may be the issue within New Relic causing this problem.

[jpvajda]

[jpvajda]

Looking at Segment I see these events firing from the docs site. I don't see an event called signup_click

[jerelmiller]

@tariqahmed @jpvajda If I remember correctly, signup_click no longer exists since we aren't instrumenting sign up clicks with NR Browser. The sign up clicks are only instrumented via Tessen.

To see the raw tessen data, I believe you'll have to switch to the Tessen account and check that way. I see the following code in our sign up link:

        tessen.track('stitchedPathLinkClick', 'DocPageLinkClick', {
          href,
          path: location.pathname,
          component: instrumentation?.component,
        });

Should be under an action as something like TDOC_DocPageLinkClick_stitchedPathLinkClick (if I'm remembering the order of how Tessen stitches that name together 😆.

cc @zstix

[jpvajda]

adding to @jerelmiller we also documented all the events and action names for clarity: https://github.com/newrelic/docs-website/wiki/Event-Data--and--Instrumentation

[jpvajda]

It's still not clear to me what GA is measuring, it's an event called Sign Up

[jpvajda]

@tariqahmed apart from my question on what is GA actually tracking(which isn't necessarily related) I think the solution to this problem is the change your queries to use the tessen events and track the stiched path events documented here

SELECT count(*) AS 'Signups' FROM TessenAction WHERE nr_product = 'DOC' AND eventName = 'TDOC_DocPageLinkClick_stitchedPathLinkClick' SINCE 1 day ago

[tariqahmed]

SELECT count(*) AS 'Signups' FROM TessenAction WHERE nr_product = 'DOC' AND eventName = 'TDOC_DocPageLinkClick_stitchedPathLinkClick' SINCE 1 day ago

^^ Shows 21

SELECT count(*) FROM PageAction where appName = 'docs.newrelic.com' WHERE actionName = 'externalLink_click' and href like '%signup%' SINCE 1 day ago

^^ Shows 71

The TessenAction's don't seem accurate. When I test by clicking on the Free Account button, I see them come in:

SELECT * FROM PageAction where appName = 'docs.newrelic.com' WHERE actionName = 'externalLink_click' and href like '%signup%' and city='Tracy' SINCE yesterday LIMIT 10

But I don't see them getting registered as a TessenAction:

SELECT * FROM TessenAction WHERE nr_product = 'DOC' AND eventName = 'TDOC_DocPageLinkClick_stitchedPathLinkClick' SINCE yesterday

Even created a new account.

I was able to eventually get it to fire by going to
https://docs.newrelic.com/docs/release-notes/agent-release-notes/php-release-notes/php-agent-9140290/

And then clicking on "Free Account"

So if you hit the home page fresh, and click on Free Account, the Tessen Action doesn't fire. But if you navigate into a subpage, and then click on it, it does.

[jpvajda]

Interesting.. thanks for deducing this for us, I think I'm going to see if @caylahamann may have some insights here as she did a lot of work on this.. but I think this is a bug in our code. cc @zstix / @polfliet

[caylahamann]

Hey everyone! some context on this:

We removed the signup_click event from NR

The reason for this is that we weren't planning on tracking it on NR, the original intention was to track it as a Tessen event since it would be used by product analytics. Additionally, we were only tracking sign up clicks in the GlobalHeader. The changes we made track sign up links on every single page, regardless of where it is, because the logic is through our Link component. Therefore, if there is a link on the page, we check if it's a signup link and track it! There was some miscommunication in the original body of work, and it was rectified right after GA.

Reasons for why the data may be off:

1. I'm seeing a super interesting error in the network tag while debugging. On the index page, when clicking on a link, we get a 414 error that the request URI is too large. You can see the event being tracked here:
   
   But then an error occuring here (the red response):
   

My guess is that somehow the tessen code is sending our page data and the data is too large? I'm not really sure what's going on. It seems to be a Tessen internal problem that we will probably have to do some further investigation on. It might also account for some of the discrepancies that we are seeing in page actions. A helpful tip to debug this is to add Tessen.debugLevel(2) somewhere in the code and then play around with Tessen actions. I can always do this, but as I'm no longer on the team definitely don't want to take work away 😄

1. The cookie consent dialog!

As mentioned previously, the cookie consent dialog (if not accepted) won't let any actions fire from tessen. We still track NR events (and we might need to reconsider if this is okay either because GDPR might not allow us too, it was little unclear in the conversation we had in the #gdpr channel, convo here: https://newrelic.slack.com/archives/C6SQHFBRQ/p1613590075008300) but we track NR events and not tessen events. I've also noticed (just from my own team) that if you're visiting a page briefly it's pretty easy to just ignore the box and never click it. You might want to look into making the cookie consent dialog box a bit more aggressive so that it's not in limbo at the bottom because users can fully use the page without choosing an option, meaning that they might not care either way. We can also look into instrumenting the cookie consent dialog box to see if it gets clicked!

[jpvajda]

Thanks @caylahamann this is all SUPER helpful. 🥂

[austin-schaefer]

Relates to #1331 for anonymous tracking info

[LizBaker]

Just wanted to chime in on what @tariqahmed was last seeing.

I am able to trigger that Tessen event from the homepage on a fresh load but I accepted the cookie first. It's possible you said no to it at some point so you'll never trigger it, or maybe when you tried with that other page you happened to click accept?

You can test this out by checking the cookie in your browser dev tools > application > cookies > newrelic-gdpr-consent. If it's false then you'll want to delete it and accept the cookie banner that will appear. If it's true and you don't see Tessen fire-- that's interesting... and we can explore that. It hasn't failed for me throughout my exploration today.

I can say with some certainty that the PageAction will always be a higher number as it tracks all signup click events via the browser agent while the Tessen event is only tracking folks who accepted the banner. Like Cayla said it's a very unobtrusive banner at the bottom so it's probably getting ignored.

As far as the error Cayla came across goes that looks to be coming from the agent and not Tessen. Hopefully just a fluke as it continually reports. I haven't seen any come up as I've been poking at it today.

So, my conclusion is that PageAction is going to give you the true count of signup clicks while Tessen will give you the consented count. As far as GDPR goes I think as long as there isn't enough information to trace these events back to a specific person it's good to go. If we add authentication to these sites we'll have to rethink this tracking as it could include emails etc.

[jpvajda]

I've reached out to @tariqahmed to see if he can review the findings above.

[tariqahmed]

@LizBaker

Yup, set to true.

[tariqahmed]

Hmm ok it appears to be working.... tested on multiple browsers, restarting fresh, etc...

So basically the difference between TessenAction and PageAction lets you derive how many people are basically not clicking on the consent.

Interesting Scenario:

What I did notice in retracing my steps - because the consent thing isn't blocking, and also innocuous, as a user you'll just go about using the site, potentially signing up... and so although you didn't say No, you didn't say Yes either... and the Tessen tracking doesn't kick in until you click on that Yes.

Thoughts/suggestions:

• Instrument the Cookie Consent dialog window so that we can track more authoritatively the choices users are making
• Run an Experiment by making the consent dialog a modal window that forces the user to make a decision, and see if that results in a higher Yes conversion rate.

@jpvajda @austin-schaefer

[jpvajda]

@tariqahmed We have an issue for the instrumentation of the dialog here: #1507 And we'll consider moving the cookie dialog banner to be more pronounced after we instrument the dialog properly. I don' think we need to run an experiment to determine the placement or if it needs to be more pronouced there are plenty of real world examples and articles on this topic, and if our goal is to get more users to click "accept" or at least click "deny" then making it more visible is a pretty easy decision.

As a note: we are also looking at enabling anonymous tracking in GA #1331 which will then allow us to track high level metrics, like sessions and page views for users that never click either button, or those that click deny. By doing this, then moving the cookie dialog banner around may just be unnecessary.