Amphtml: I2I: amp-funnel-relay

[trackonomics]

Summary

Funnel relay is a tool for publishers in the affiliate market. The tool involve a script running on articles's pages, extends affiliate links with extra information and track events on analytics that are later merged with other information about the traffic.

Code:
https://github.com/trackonomics/amphtml

Design document

https://docs.google.com/document/d/1j7p9Mgh8dvQu6JwmZ4WN6Ww8KjCRqjoXyAMt7ZuuA0E/edit?usp=sharing

Motivation

Publishers uses funnel-relay script on their website in order to extend the data collected of publisher’s traffic. Since AMP does not allow 3’rd party script it is required to create custom tag to allow publishers to run funnel-relay on their amp page.

Since funnel-relay presents unique logic used by Trackonomics' customers, there is no other alternative among AMP's custom tags.

Additional context

/cc @ampproject/wg-approvers
/cc @lannka

[cathyxz]

/to @zhouyx and @ampproject/wg-analytics since @lannka is out

[torch2424]

Thanks @cathyxz , @zhouyx , did Hongfei leave any last notes on the Affiliate linking extensions before they left? What's our plan forward for this? 😄

[zhouyx]

Hello @trackonomics. Thank you for your interest in integrating with AMP.
We recently get a list of similar requests. Instead of having every service provider add their own component, we think a more general support can best save development and maintenance effort. Please take a look at #20653 and let us know if the general support can fulfill your requirement.

[trackonomics]

Hello, @zhouyx thank you for your replay.
Our solution is unique and in the future we are planing to do a very unique interaction with web-service regarding link behaviour. Unfortunately we can not use general tag.
Even if there is a general tag for extending link attribute, and/or replacing link HREF value, we will need another custom tag, to cause our settings to interact and configure this tag. I can't see how do we or other vendor avoid from having custom tag to deal with specific vendor's logic.

Regards
Gilad.

[zhouyx]

@trackonomics Thank you for explaining.

I understand there're some restrictions of the generic solution. It's regex based which requires you to find all rewriting link using regex. And it's a pure client side solution.

I'm trying to better understand your solution. I have a few questions

1. How do you decide which link on the page to rewrite?
2. Is there a configuration to define the behavior
3. How to configure the format of the rewrite link?
4. Are all computation done client side, or do you rely on server side input

Our idea is still to introduce a generic solution. Please let us know so we can look into expanding the current design to support your use case.

FYI: The team won't have bandwidth to look into it before the end the April (AMP Conf!). Thank you!

[trackonomics]

Hello,
Below my answers to your questions.

Q: How do you decide which link on the page to rewrite?
A: Depending on detection rules set by client. the rules are stored in client's specific settings file loaded from server.

Q: Is there a configuration to define the behavior
A: Yes, We have special tool we give to publishers. They use this tool to configure the behaviour of the funnel-relay script. This configuration include information about, what kind of information we want to add to link, what detection rules to use etc'. When customer save settings, a file is created on our server. AMP tag read this settings file, based on some client's specific code. Then it operate some behaviour based on customer's

Q: How to configure the format of the rewrite link?
A: The configuration of rewrite is not handled by the AMP tag, nor by the Javascript code run on publisher's page for non-amp pages. In fact, the configuration part is out of scope regarding the amp tag we want to you to approve. The AMP tag, implements some logic based on configuration, but it does not take part in this configuration.

Q: Are all computation done client side, or do you rely on server side input
A: Computations rely on settings data loaded from server.

The bottom line , from software engineering point of view, is that funnel-relay script, for which we need AMP tag, is part of broader system. This system includes both client and server side software modules. Some of these modules are beyond the amp tag scope. Publishers use some tool to configure behaviour of script, later run on their website. The script therefore implements some customer logic based on some custom data. This can't be resolved by generic tag. Generally, custom business logic can't be implements by generic components. There is a always need for some custom code. Therefore in order to support our and others custom logic, there is no other way but providing a way to implement custom AMP tags. Otherwise publishers will not be able to use custom solutions, and will not be able to enjoy the value of such solutions. I understand the need to reduce code, but at the same time I don't think AMP need to be a barrier for publishers to enjoy the value brought by custom solutions. custom solutions requires custom tags.

Regards,
Gilad.

[torch2424]

@trackonomics Hello! Thanks for the response. Just a heads up on next steps, if the solution we recently released does not work for you, we may need to create a new extension, which has a next steps 😄

So what you will want to do is sign up for a Design Review, and unfortunatel, all of us on the team handling link rewrites currently is going to be preparing, or on vacation around our AMP Conf event for April.

Thus, you may want to sign up for one of the Design Reviews that work for you in May.

Just to get a bit more background on this in the meantime. It seems like from what you described, most of the re-writing happens on the server correct? Is the client side implementation simply interpreting this returned server config? What stops you from translating your JSON into one that's compatible with amp-link-rewriter and embedding it on the AMP pages? Is some of this server side logic not handled by you? Is there public facing documentation I can look at to help understand?

Thanks! 😄 👍

[trackonomics]

Thank you. The code was already created and is on github. I will apply to Design review.
As for your question, not exactly - the settings is stored on the server. The funnel-relay script uses this settings to manipulate link on page. This also include recording analytics events.
We do not process links on the server, and in fact we try to avoid doing so.
Also out tag will need the 100% capabilities of custom solution, since we going to extend the way we work on links in the future.

As I mentioned, publishers that will use this tag are customers of Trackonomics LTD. They have good reason to use this tag because it bring and it will bring value for the,and will bring much more value in the future. All we want is to give them the choice to enjoy this value on AMP pages. My perspective is that AMP should not be the barrier for customers to get value from service vendors. Custom tag with full freedom to apply or logic is perfectly fine for us.

Regards,
Gilad.

[trackonomics]

Hello

I have a question. I probably did not understand correctly the procedure.
You suggested me to submit to design-review, but instead I went and open I2S issue. Probably a mistake.

https://github.com/ampproject/amphtml/issues/21755

Anyway, what do I have to do now, in order for amp-funnel-relay to be approved?
Is Design Review, a mandatory part of the procedure?

Can I move already to phase of "Create a Pull Reques" or do I have to wait for something else?

Regards,
Gilad.

/cc tourch2424

[torch2424]

@trackonomics

As for your question, not exactly - the settings is stored on the server. The funnel-relay script uses this settings to manipulate link on page. This also include recording analytics events.
We do not process links on the server, and in fact we try to avoid doing so.
Also out tag will need the 100% capabilities of custom solution, since we going to extend the way we work on links in the future.

Ah, thanks for the explanation. Could you perhaps provide some more documentation, or an expanded design doc so I can get more of a visual. If we end up doing a design review this will be very helpful then as well. 👍 See the amp-skimlinks and amp-smartlinks as an example. That being said, if you could explain your reasoning similar to these two. It would be helpful for us to identify the common denominator in your solution to the other custom solutions, in helping us determine why a custom solution is best here, or how we can make another generic solution that fits a use case similar to yours. 😄

I have a question. I probably did not understand correctly the procedure.
You suggested me to submit to design-review, but instead I went and open I2S issue. Probably a mistake. #21755

Thanks for opening that! No worries, I don't think an Intent-to-ship is needed for this, as we are not shipping this yet, and we still have to make the changes.

Anyway, what do I have to do now, in order for amp-funnel-relay to be approved?
Is Design Review, a mandatory part of the procedure? Can I move already to phase of "Create a Pull Reques" or do I have to wait for something else?

So I talked with another engineer on the team, I think it would be fine for just our analytics working group to look this over. But because of the AMP Conf like I mentioned earlier, a lot of us are out of office (e.g @lannka @zhouyx ). So it may be a little while until we all get to this. You could technically open a pull request, but I would personally advise against it, in case we (as in you and our working group) come to a conclusion (after talking with you and learning more about your design) we need to change directions on the overall design of the extension. 🤔 👍

Let me know if this helps, or you have anymore questions! 😄

[trackonomics]

Thank you.
I will wait for May then.

I will work on extending this design document during this time. Basically we need the same freedom we have with regular web page (without AMP). This include:

1. Extending link URL
2. Replacing link URL
3. Detecting events on page and recording to GA or other analytics
4. DOM Manipulation (for our more advanced solution)

I will make these things reflect in the design doc better.

https://docs.google.com/document/d/1j7p9Mgh8dvQu6JwmZ4WN6Ww8KjCRqjoXyAMt7ZuuA0E/edit?usp=sharing

[lannka]

@trackonomics could you enable the comment permission to your doc so that we can move the discussion to your doc?

In general, 1) & 2) are common requirements across different affiliate vendors and are well implemented in amp-link-rewriter extension. If you see anything missing, we can address them.
For 3), we'd like to recommend the use of amp-analytics.
For 4), I'd like to hear more details about what DOM manipulation are you thinking of.

Overall, as @zhouyx & @torch2424 mentioned, we're in favor of reusing components to keep AMP usable and maintainable.

[trackonomics]

@lannka

Hello, I hope you’re well and the conference went great. Here
at Trackonomics we’re eager to make progress with amp-funnel-relay tag.

I completely understand that you want to reduce the number of tags on AMP in order to ensure a generally high level of quality assurance. Naturally, you need to balance that approach with the fact that publishers that are using AMP have particular tracking needs that are provided by different companies / technologies.
At Trackonomics, we are proud to count as clients some of the largest and most prestigious publishers in the world, including CBS, NBC, Buzzfeed, Turner Media, CondeNast, Meredith, NewsUK, and many many others. They all rely on Trackonomics to provide our data services across their assets and platforms, including on AMP, and this can only be achieved with a dedicated AMP tag of our own.
The main reason is the need for total flexibility in controlling the tag logic. In addition we use our own events recording service, which is not supported by any other tag. Please find below a link to an updated document, where you can also
comment:

https://docs.google.com/document/d/1j7p9Mgh8dvQu6JwmZ4WN6Ww8KjCRqjoXyAMt7ZuuA0E/edit?usp=sharing

We’re happy to discuss, expand and explain.
But please do not dismiss our claims and requests idly.

Kind regards,
Gilad.

[lannka]

Left some comments to your doc, and we can chat there. In general, so far I still don't see a big reason why the features can not be done by amp-link-rewriter + amp-analytics.

[trackonomics]

Thank you for your comments. Please find my reply on the design doc.
amp-link-rewriter can't be a replacement. For example:

1. It can't read metadata from the page DOM and embed it on links or send to Analytics.

2. It does not have custom detection rules that can be change by customer, on-demand. It don't have page level filter, It don't have turn detection rule on/off. It uses CSS selectors to detect anchor element to manipulate, and this is not a reliable approach to detect elements, since CSS may vary occasionally.

3. It don't have a way for customer to decide which of the page URL parameters, will be added to link and/or recorded

4. It can't write to our own Events-recorder.

5. Can it share per/click UUID with analytics-tag, so that same UUID appended on link when user click, is also recorded as primary-key on analytics record?

6. Does it have exclusion rules?

7. Does it know how to handle links created dynamically by api such as Viglinks and Skimlinks?

8. It can't collect information about various timing and other events occur on page. It can't collect information about user agent.

9. It can't produce a report of activity that we use in our funnel-relay simulator.

10. And more....

I think we discussed a lot. I want to make progress. I need to know if the community is about to approve to move forward with funnel-relay tag.

as I explain in my comments on design doc, the KIS (keep-it-simple) principle in this case, is simply using custom tag. amp-link-rewriter may help us to provide very basic solution, but this solution does not cover all the functionality we need and we may. The result of that is that we will have to maintain two lines of products:

• 2 X Client for setting funnel-relay
• 2 X Javascript
• 2 X Data collection (since AMP can't give all the information we need)
• 2 X Technical document
• 2 X Training and supporting customers

One for AMP pages, another for non-AMP . This make no sense in terms of our available resources.

As I mention in my comments on design doc, the best solution for AMP, not to deal with endless requirements for custom tags, is simply create tag that let you load and run custom Javascript. This is the website owner that need to decide what will run on the website, not AMP developer community.

Regards,
Gilad.

[torch2424]

@lannka Can we provide an update on this? 😄 👍

[lannka]

The discussion was moved to the design doc. Is this something you still plan to work on? We can schedule an office hour to address the questions & concerns.