Amphtml: [Discussion] <amp-consent> TCFv2 Support

[zhouyx]

The Transparency and Consent Framework (TCF) is an effort to manage user consents, and have all parties understand the users consent information from regulations like GDPR.

Consent Management Platforms (CMP) helps sites to collect and manage user consent information in the TC data format. Today AMP allows a CMP to integrate with some limitations.
The <amp-consent> CMP integration was designed to work with any framework that is based on a consent string format.

CMP scripts are executed in an CORS iframe that’s controlled by AMP instead of included as first party script. Because AMP handles the lifecycle of the iframe, CMPs don’t have flexibility on when to show its UI, or has the ability to provide mandatory APIs for vendors to get their TC data.

What has Changed

In the past year, the IAB TCF v2.0 has been widely adopted by most vendors. Unlike a year ago when we first designed the feature, now we don’t see the necessity to support other TC framework. This allows AMP to work together with CMPs to understand the TC data and TCF v2 APIs.

With more regulations like GDPR, we are seeing the increasing complexity for site owners to manage user consents. Thus we are seeing the trend to have more CMPs come into the area. That brings a lot more feature requests and many ask for feature parity between AMP and their non-AMP integrations.

Goal

The goal of the design revise is to come up with a solution so that

1. Vendors don’t find integrating with AMP challenging due to user consent handling.
2. CMPs and publishers get flexibility to manage consent prompt without hurting general user experience.
3. AMP pages are still performant.
4. AMP doesn’t take the liability in managing user consent data.
5. Engineering effort is manageable.

Proposed solutions

Option 1

Allow CMP script to run in the background, it will still run in a CORS iframe, but it will be initiated at pageload time and always run in the background.
This is very similar to the CMP, non-AMP pages integration as much as possible.

Details

• Different ad vendors will have different ways to call the TCF standard APIs
  
  
  
  • Fast Fetch: AMP can proxy the __tcfapi() method and send postMessage to CMP iframe
  
  
  • In iframe: Same with non AMP, Let vendor communicate with CMP iframe __tcfapiLocator directly using postMessages.
  
  
  • Analytics vendors can’t use the TCF standard APIs. AMP can provide macros for them to get vendor specific data from the TCData object. But we will start with the entire consent string.
  
  

Pros:

• Vendors can follow the same implementations, as the same set of APIs will still be supported by CMPs.
• Because the CMP iframe runs in the background. It has the flexibility to request to display the UI.
• Easy implementation from the AMP side. As CMPs have all APIs provided, AMP simply passes messages between different parties.
• This may also enable publishers to work together with CMP to manage vendors that’s not only the GVL list, as long as the two parties reach some agreement on format. (Need declarative inline config added)

Cons

• Having CORS iframe running in the background can lead to performance and security concerns.

Option 2

This is the current approach. Pass the entire TC data consent string using AMP provided APIs

Design Details

• AMP will provide APIs for vendors to get the complete TC data in the consent string format.
  
  
  
  • 3P Ad vendors can use the context.initialConsentValue
  
  
  • Fast fetch vendors can use the getConsentPolicyInfo() method
  
  
  • Analytics vendor and RTC request can use the CONSENT_STRING macro
  
  

Pros

• No extra work on the AMP size. Support future version of the framework by default.
• Passing TC data and handling them on the server end is the best for performance.

Cons

• CMPs don’t get flexibility on the consent UI.
• Difficulties for vendors to interpret TC data on the client side.
• Ads vendors need to import libraries to do so. This can be a blocker for future vendors to integrate.

To decide if we need to change the design, we'd like to get feedback from CMPs, Ad vendors, Analytics vendors and Publishers.
We are leaning towards Option 2 (The existing option) for performance reason. If so

1. How difficult it is for ad vendors to interpret consent string without the TCF APIs
2. Are analytics vendors OK with handling the consent string on server side?
3. What's the limitation to CMPs having AMP runs their code in CORs iframe, and controls the lifecycle of the iframe?

cc @ampproject/wg-analytics @ampproject/wg-ads

[zhouyx]

@AmazonPublisherServices @leonardlabat @bretg @wroberts-comscore for feedback since you have dealt with CONSENT_STATE and CONSENT_STRING in AMP.

@yoannOgury @PierigOgury @andresilveirah @jawadst @tla-sirdata for feedback from CMP

Thank you!

[bretg]

Prebid Server receives RTC calls from AMP. We find it sub-optimal to have to introspect CONSENT_STRING, which could be formatted as a TCF1.0 or CCPA string.

Would it be possible to add a "CONSENT_TYPE" attribute which the CMP can add to provide a clear signal on how the string is formatted? Valid CONSENT_TYPE values could be: TCF1, TCF2, USP, etc.

How difficult it is for ad vendors to interpret consent string without the TCF APIs

Speaking for Prebid, if we had CONSENT_TYPE, we'd be happier parsing strings. We do it already in Prebid Server. In fact, we don't currently use the AMP CONSENT_STATE -- we assume that the user consent state is already encoded in the string.

It's unsatisfying and won't scale well to parse the consent string with a set of rules like:

• if it's 4 chars and starts with a "1" it must be CCPA
• else if it passes through the TCF1.0 parsing function, it must be TCF1.0
• else if it passes through the TCF2.0 parsing function, it must be TCF2.0
• else it's an error

[zhouyx]

Thank you @bretg for the feedback.
TCF1.0 and 2.0 should be easy to interpret as the version number is included in the TC string. But I agree that the US privacy string make things more difficult.
Is there a standard name that the AMP team can follow? Could you please let us know how CMPs are passing this information in non AMP page today? Thank you

[bretg]

In non-AMP pages, there are different IAB functions in the page that serve as the interface. For GDPR, the ad tech javascript infrastructure calls window.__cmp(). For CCPA it's window.__uspapi(). So because these separate interfaces exist, it's quite clear to the ad ecosystem how to parse the results of the call.

Being on the vendor end of RTC is quite different - we're not making the call to the CMP ourselves so don't have the context.

FWIW - prebid.js requires that publishers turn on GDPR and USPrivacy and let it know which specific interfaces to use. e.g.

pbjs.setConfig({
         consentManagement: {
           gdpr: {
            cmpApi: 'iab',  // could also be passed in statically
            ...
           },
           usp: {
             cmpApi: 'iab',  // could also be passed in statically
             ...
           }
         }
       });

[PierigOgury]

As a CMP first option seems attractive, it will give us more flexibility, each vendors will be able to use the exposed api or the consent string, without any specific intégration for AMP, it is the idea behind tcf implementation.
This option also allows the most limited liability on AMP side.

On AMP side, the second option will need less engineering and is clearly more safe technically, and dealing with CONSENT_TYPE seems a good simple solution .

However it will force vendors to find a solution to be able to decode consent string, with one decoder by consent string type, it could be really painful for vendors, with some performances impact.

Thats why as a CMP, we prefer the first option, but the second one works also with more impacts for vendors.

In all cases, we think that AMP have to keep the hand on which kind of consent string can be provided through the amp-consent.

At the same level as the CONSENT_TYPE remark, it would also be useful to be able to set multiple consent signals (in case of a CMP managing multiple framework/regulation with one notice to the end user) 

[bretg]

dealing with CONSENT_TYPE seems a good simple solution. However it will force vendors to find a solution to be able to decode consent string, with one decoder by consent string type

Not quite sure I understand you here @PierigOgury - are you suggesting that the existence of CONSENT_TYPE makes parsing _harder_? My take is that each RTC vendor has to have code to parse all encodings anyhow... having metadata about the encoding format just means no guessing.

[PierigOgury]

Hi @bretg ,
I mean that the first solution proposed allows to not have a decoder in case the API gives an "easier" to manage information (such as the TCF API regardless of the version). With CCPA framework it is mandatory to have a "decoder" as the API is sending only the encoded part.
With the second solution a decoder is mandatory regardless of the API defined by the framework, for anyone in the amp environment (vendor and publisher). And as you mention, the CONSENT_TYPE allows for no guessing on the type of object received.

[zhouyx]

Thanks all for the feedback.

After more discussion, the AMP team decided Option #1 is not very practical. As @PierigOgury pointed out it requires a lot engineering resources. Also it's not technically safe because AMP is not a registered CMP. Because of this we decided to stay with the current approach (Option #2)

[zhouyx]

Re: CONSENT_TYPE. Sounds reasonable to add the additional information.

I only have one concern, the CONSENT_TYPE only makes sense when it is stored with consent string together and can be understand by all parties. @PierigOgury @bretg does this mean that AMP needs to declare the types of consent string framework we support and provide a list of standard type names?

[tla-sirdata]

Hi,
First sorry for my late feedback.
I would like to bring more elements even if it seems the AMP team has made its decision.

I think the fact CMPs have no control on whether to display the UI is a problem.
When the vendor list changes, we need to display the UI to ask consent for additional vendors while keeping previous choices. With the actual integration, we can only expire the consent string, which is obviously not what we want.
TCF v2.0 also introduces a TCF Policy version that we have to take in consideration to display the UI when it changes.
Additionally, the recent CNIL notice (still in public comment) indicates that the UI should be displayed again in case of "significant change".
These are just some examples but we can see there can be many reasons why CMPs need to display the UI, and they change over time depending regulations and frameworks.
If AMP keeps control on the display, it must ensures those rules will be applied, which I think is painful for AMP engineers and doesn't make sense as AMP is not a CMP.

As a reasonable solution, CMPs could add an attribute in the check endpoint's response to indicate to AMP that we want to display the UI while keeping the existing consent string.
On AMP side, a capping could be managed to prevent CMPs from displaying the UI too often.
This solution doesn't require a CORS iframe.

---

About reading consent data, it would be indeed very convenient for vendors to call the standard APIs and not have to decode consent strings.
Actually in TCF, vendors are not supposed to decode the consent string by themselves, they should always read consents through the CMP API, which ensures them that signals are valid and the consent string is not corrupted.
That point should probably be discussed with the IAB Europe as they probably didn't design the framework with AMP concerns in mind.

The limitation I see in the actual integration is that CMPs can only transmit a consent string (and maybe a future consent type).
As mentioned in the discussion's description, publishers may need to work with vendors outside the GVL list (example: Google before TCF v2 adoption), so we have to transmit those additional information.

The main idea is to provide CMPs a way to transmit more information (at page load and after user submit), the challenge being to give enough flexibility to CMPs while keeping a standardized AMP structure.
As a workaround to standard APIs, CMPs could then use that feature to expose decoded data to vendors.
Once again no CORS iframe is needed.

[zhouyx]

Talked to @jeffjose offline. We agreed that AMP can provide such CONSENT_TYPE information. TCF v1, v2 and US privacy string will be the three formats we support for now. In the future we are open to add other format to the list upon requests.
The team is currently focus on adding us-ca detection capability to amp-geo, and doesn't have bandwidth to this API this sprint. We'll update the thread once we pick up the work. Thanks.

[zhouyx]

As a reasonable solution, CMPs could add an attribute in the check endpoint's response to indicate to AMP that we want to display the UI while keeping the existing consent string.
On AMP side, a capping could be managed to prevent CMPs from displaying the UI too often.
This solution doesn't require a CORS iframe.

The only concern is user experience. @micajuine-ho @jeffjose and @lannka would like to get your thoughts on the tradeoff here.

The main idea is to provide CMPs a way to transmit more information (at page load and after user submit), the challenge being to give enough flexibility to CMPs while keeping a standardized AMP structure.
As a workaround to standard APIs, CMPs could then use that feature to expose decoded data to vendors.
Once again no CORS iframe is needed.

We could provide an API for CMP to pass consent data object to vendors. AMP won't store the data
object or enforce the object format. My concern is that without standard structure, such type of object cannot be parsed by vendors.

[tla-sirdata]

The only concern is user experience. @micajuine-ho @jeffjose and @lannka would like to get your thoughts on the tradeoff here.

Sorry @zhouyx, are you asking me to give my thoughts about the tradeoff here ?

[cmaurersp]

@zhouyx wanted to see where the AMP support for TCF version 2 stands. i'm having trouble understanding it's progress from this thread.

[jawadst]

@zhouyx Having worked with many vendors in the last months on TCFv1 integration with AMP, the main piece of feedback we have is that most vendors are still unclear on how they can get access to the consent string from their AMP tags.

In particular, they are struggling with two elements:

• What tags can get access to the consent string and how (macros, usually)
• How the different parameters on ad tags affect the availability of the consent string or the loading of tags

We have tried to summarize all common cases on https://developers.didomi.io/cmp/amp/consent-status-for-vendors and https://developers.didomi.io/cmp/amp/blocking-behaviors but I'd recommend improving the AMP docs for vendors on that front for the V2. Happy to help if there is a way for us to contribute to the docs.

[zhouyx]

Thank you @jawadst This is very helpful! I've added you as reviewer to #27855

[zhouyx]

@cmaurersp any specific detail you need?

AMP attempts to support TCF v2 as a generic TC framework as best as we can.
AMP will collect and store raw consent string from CMP and pass the raw string value to vendors.
As described above AMP decided not to support the TCF v2 APIs though, and expect vendors to parse the string on the server side.

[cmaurersp]

@cmaurersp any specific detail you need?

AMP attempts to support TCF v2 as a generic TC framework as best as we can.
AMP will collect and store raw consent string from CMP and pass the raw string value to vendors.
As described above AMP decided not to support the TCF v2 APIs though, and expect vendors to parse the string on the server side.

What do 3rd party CMPs need to do in order to support TCF v2 at this point in time?

[zhouyx]

If you want to integrate your CMP solution with AMP. For example

<amp-consent type='myCMP'></amp-consent>

Please follow the CMP integration guideline here. https://github.com/ampproject/amphtml/blob/master/extensions/amp-consent/integrating-consent.md
Let me know if there's any question.

[saraneus]

We are creating a CMP for our publishers and would like to do an integration test locally before we actually create a pull request. How can we create a full working demo?

Our demo looks like the following so far:

<amp-consent id="myUserConsent" layout="nodisplay">
    <script type="application/json">{
        "consentInstanceId": "consent-id",
        "consentRequired": true,
        "promptUIaaa": "consentDialog",
        "postPromptUI": "post-consent-ui",
        "checkConsentHref": "https://open-cmp-local-api.de:5003/consent/check",
        "onUpdateHref": "https://open-cmp-local-api.de:5003/consent/update",
        "promptUISrc": "https://open-cmp-local-iframe.de:5003/demo/amp/cmp-frame.html",
        "storageKey": "cmp_foo_bar"
    }</script>
    <div id="post-consent-ui">
        <button on="tap:myUserConsent.prompt()">Update Consent</button>
    </div>
</amp-consent>

In Dev tools I can see a iframe being created loading promptUISrc, which is the standalone CMP. No errors reported. But the iframe is hidden and only a spinner is displayed. It's probably something I missed, could someone give me a hint?

[tla-sirdata]

Hi we would like to integrate our TCF v2 CMP to AMP and have some questions :

• Can you provide information about integration and the new CONSENT_TYPE field ?
• How are we supposed to add the configuration here ?
• Is there something to indicate v2 endpoints or should we create another config type (for example : sirdata-v2) ? Same question for CCPA
• TCF v2 requires a very large amount of information on the 1st layer, which can result to a dense content with the 60vh restriction. Can we consider increase it ?

Thanks

[micajuine-ho]

Hi @saraneus. Thanks for your question. Your current implementation of adding your CMP to AMP seems a little off. To integrate into AMP as a CMP, you need to follow the CMP integration guideline here. https://github.com/ampproject/amphtml/blob/master/extensions/amp-consent/integrating-consent.md

After following these steps, you can test locally with a consent element similar to this (replacing _ping_ with your CMP name):

<amp-consent id='ABC' layout='nodisplay' type='_ping_'>
      <script type="application/json">
      {
        "postPromptUI": "postPromptUI",
        "clientConfig": {
          "CMP_id": "test_id",
          "other_info": "test_info"
        }
      }
      </script>
      <div id="postPromptUI">
        Post Prompt UI
        <button on="tap:ABC.prompt(consent=_ping_)" role="button">Manage</button>
      </div>

Another hint is that

        "checkConsentHref": "https://open-cmp-local-api.de:5003/consent/check",
        "onUpdateHref": "https://open-cmp-local-api.de:5003/consent/update",
        "promptUISrc": "https://open-cmp-local-iframe.de:5003/demo/amp/cmp-frame.html",

should be here under your new CMP name.

Please let me know if you have any further questions.

[micajuine-ho]

Hi @tla-sirdata, thank you for your questions.

Can you provide information about integration and the new CONSENT_TYPE field ?

This is still in the works. Please refer to #27907, and we can continue the discussion there.

How are we supposed to add the configuration here ?

To integrate into AMP as a CMP, you need to follow the CMP integration guideline here. https://github.com/ampproject/amphtml/blob/master/extensions/amp-consent/integrating-consent.md
If you have any questions on these steps, please let me know.

Is there something to indicate v2 endpoints or should we create another config type (for example : sirdata-v2) ? Same question for CCPA

I believe you are asking if there is someway to tell if the consent_string you are passing along through your endpoint satisfies TCF_V1, TCF_V2, or US Privacy String. If this is the case, then I believe that this issue can be solved with the addition of the CONSENT_TYPE field as metadata for the consent string (again, see the issue linked above).

TCF v2 requires a very large amount of information on the 1st layer, which can result to a dense content with the 60vh restriction. Can we consider increase it ?

@zhouyx correct me if I'm wrong, but I believe we decided on 60vh restriction due to the bad UX that would be created if users were to just see a fullscreen consent dialog. However, you can use our fullscreen API to enlarge the consent dialog.

[tla-sirdata]

@micajuine-ho Thanks for your answers !

I believe you are asking if there is someway to tell if the consent_string you are passing along through your endpoint satisfies TCF_V1, TCF_V2, or US Privacy String. If this is the case, then I believe that this issue can be solved with the addition of the CONSENT_TYPE field as metadata for the consent string (again, see the issue linked above).

I'm talking about the CMP version a publisher chooses to use.
Actually we're already integrated in AMP as a TCF v1 CMP and would like to add our TCF v2 version. I'm wondering if publishers will have to change the <amp-consent> type (from sirdata to sirdata-v2) or if they will be able to switch with a parameter in the JSON config.

Thanks

[micajuine-ho]

@tla-sirdata Thanks for the clarification.

As a CMP do you anticipate that having publishers using v1 and v2 will be a common use case for you?

Something that we could do would be to include a CONSENT_TYPE field within the request to the CMP (in the checkConsentHref endpoint), that could be overriden by the inline publisher, but would otherwise default to the last CONSENT_TYPE stored in local storage.

I don't think we currently have a clear cut solution for this. @zhouyx for ideas/confirmation.

[zhouyx]

I have similar questions.

1. Who decide which TCF version to support? Publisher or the CMP.
2. For your integration, will you hit the same endpoint and provide the same UI (via same url) for TCF v1 and v2 support?

[tla-sirdata]

1. The publisher decides when he chooses to switch to TCF v2 when he's ready to.
2. We've developed two new endpoints (check + UI) for the TCF v2 so both versions could live in parallel. We could use the same endpoints to serve both versions but in this case we'll need to update our current endpoints.

Also I don't know how AMP will deal with CCPA but we were thinking about detect geolocation through our v2 check endpoint to provide the good UI (CCPA or TCF v2). Does it make sense to you ?

[zhouyx]

amp-geo provides the geolocation support via preset-us-ca and preset-eea

We've developed two new endpoints (check + UI) for the TCF v2 so both versions could live in parallel. We could use the same endpoints to serve both versions but in this case we'll need to update our current endpoints.

It will work both ways. If you can reuse the same endpoint, then you can ask publishers to choose the TCF via the clientConfig. If you want to have two endpoints, then you can integrate sirdata-v2 with us as a new CMP. But since the TCF v1 support will be deprecated soon, please let us know your deprecation plan with the existing sirdata integration. Thank.

[tla-sirdata]

amp-geo provides the geolocation support via preset-us-ca and preset-eea

OK so it will be up to the publisher to implement geolocation and choose CCPA or GDPR.

It will work both ways. If you can reuse the same endpoint, then you can ask publishers to choose the TCF via the clientConfig. If you want to have two endpoints, then you can integrate sirdata-v2 with us as a new CMP. But since the TCF v1 support will be deprecated soon, please let us know your deprecation plan with the existing sirdata integration. Thank.

I prefer not to multiply CMPs. Passing the version via the clientConfig sounds good to me as both endpoints receive it. We'll dig on this.

Thank you!

[tla-sirdata]

Hi I'm testing a TCF v2 integration and have the following error when I give consent :

log.js:251 [CONSENT-STATE-MANAGER] Cannot store consentString which length exceeds 200. Previous stored consentInfo will be cleared

Here is the consent string sent to AMP : COykPRuOykPRuBcABBENAjCsAP_AAH_AAAqIF5EX2S5OI2tjI2YdF5BEYYwfJxyigMgChgQIsS4NwIeFbBoGL2AAHBG4JCQAGBAEEACBAQIkHGBcCQAAgIgBiRCMQEmMCzNKBJJAggEbM0FACCVmHkHSmZCY7064O__TAvIgkwVJwCRIUAgBIsQIhDABAOIAABgCUEAAECFhoACAjYFAR6gAAAIDAACAAAAEEAAAIABAAAgIgAAAEBACACIBAAAAEaAhAABIEAEAJEgAAAQA0IACCIIQBCDgQCjhAAA

As you can check here, the TC String is valid for TCF V2 so the consent string length should be allowed to exceed 200 in AMP.

Thanks

[zhouyx]

Thanks for raising the issue. Yes, there's plan to expand the storage limit. We are still determining the desired storage space.
More context on the AMP localstorage policy can be found here.

[micajuine-ho]

Hi all! Currently, consent dialogs can be configured to a max of 60vh, and be scrollable/expandable.

I wanted to get your opinions on if this current setup is sufficient to display all disclosures required by TCFv2, as well as the accept/reject buttons.

/cc @yoannOgury @PierigOgury @andresilveirah @jawadst @tla-sirdata for feedback from CMP

[tla-sirdata]

Hi, if we want to keep the UI as readable as possible for the user, 60vh is definetly not sufficient considering the amount of information CMPs have to disclose with the TCF v2.
In my opinion making it scrollable is bad for user experience so we're not considering this option.

Another thing is that it might be useful to allow CMPs to exit fullscreen mode so the initial layer would always be displayed the same way (after a click on a Back button for example).

Thanks

[micajuine-ho]

We have some ideas to improve the UI for the consent dialog that we are taking to design review #28108 tomorrow. Please feel free to join or once we get feedback from the whole amp team, we can discuss further.

[tla-sirdata]

Hi @micajuine-ho do you have any update on the UI ? Thanks

[micajuine-ho]

Hi @tla-sirdata, we took our design proposals to design review last week, meeting notes can be found here.

We decided that option three (Immediate Lightbox iframe) is how we would like to proceed. Next steps will be to design an API for this lightbox consent. Please let me know if you have any feedback, questions, or concerns.

[tla-sirdata]

HI @micajuine-ho thank you, option 3 looks great to me.

I have some questions :

• do this option will replace the current one or both will be available ?
• have you thought about my request to allow CMPs to quit the fullscreen mode after being triggered ?
• it appears that some publishers now develop their websites with AMP only, including for desktop. In this case amp-consent doesn't fit well (especially because of the 60vh fixed height). What do you think about it ? Is this a common case ? If so, we should consider a responsive design for amp-consent or allow CMPs to access the window width to adapt their design.

Thanks !

[micajuine-ho]

@tla-sirdata Thanks for the questions!

do this option will replace the current one or both will be available ?

We will design for both to be available, so that it is backwards compatible with older consent frameworks.

have you thought about my request to allow CMPs to quit the fullscreen mode after being triggered ?

For the immediate lightbox design, we are thinking to not allow the fullscreen API; in the common mobile case; the disclosure will take up as much space as possible (only limited by our lightbox margin), and so going from ~90% of the screen to 100% wouldn't really do much for the user.

Once the lightbox flow has been implemented, would you see a need for the 'old' flow to include an action trigger to shrink from fullscreen to original height?

it appears that some publishers now develop their websites with AMP only, including for desktop. In this case amp-consent doesn't fit well (especially because of the 60vh fixed height). What do you think about it ? Is this a common case ? If so, we should consider a responsive design for amp-consent or allow CMPs to access the window width to adapt their design.

Good point! Question: for desktop, amp-consent doesn't fit well because 60vh is too small?

If so, my impression is that the new lightbox design could help solve this issue. Definitely a scenario worth considering when we are implementing.

[tla-sirdata]

Once the lightbox flow has been implemented, would you see a need for the 'old' flow to include an action trigger to shrink from fullscreen to original height?

I think it depends the schedule because we will probably use the new lightbox flow for our TCF v2 version if available on time. In any case, we obviously won't apply the new lightbox flow to our TCF v1 version so my request is mainly for this one. But I think if the old flow remains an option, that feature would be useful for UX.

Good point! Question: for desktop, amp-consent doesn't fit well because 60vh is too small?

60vh is too big for desktop. You can see one of our publishers having integrated our CMP here. Definitely doesn't look great on desktop :) I can adjust CSS within the iframe but since I can't access the window width, I'm not able to set the right amp-consent's initialHeight (something like 30vh should be fine).

Thanks

[micajuine-ho]

But I think if the old flow remains an option, that feature would be useful for UX.

I see, that sounds good. It sounds like a reasonable request, I will see how we can plan around this work.

I'm not able to set the right amp-consent's initialHeight (something like 30vh should be fine)

You should be able to set the initialHeight to 30vh. Are you having trouble doing so?
When you send the ready signal to the AMP parent doc from your consent iframe, you can add the initialHeight field to 30vh (see here for an example).

[tla-sirdata]

You should be able to set the initialHeight to 30vh. Are you having trouble doing so?
When you send the ready signal to the AMP parent doc from your consent iframe, you can add the initialHeight field to 30vh (see here for an example).

Yes I can, that's what I do when I set the 60vh initialHeight. But I would like to set the good initialHeight depending on the window width, for example 60vh for mobile and 30vh for desktop.

[micajuine-ho]

I would like to set the good initialHeight depending on the window width

Ahhh I understand. Thanks for the clarification. Not sure if we have something like this/how feasible this would be to add to amp-consent. I will look into it.

Also, @tla-sirdata, for the new Lightbox flow/TCFv2 support, do you imagine that you will upgrade all of your publisher's pages to it? Or would it be up to the publisher or some other factor? Essentially I am trying to determine who chooses what consent framework is used for each page.

[tla-sirdata]

If we manage to design a nice and readable UI with the old flow despite all information we have to disclose, I think we'll provide publishers with the two flows and it will be up to them to choose. And on the first place we will decide the default flow (probably the lightbox flow).
About the transition between TCFv1 and TCFv2, it will also be up to publisher to upgrade as it requires their partners to be ready too.

[vyoungnyc]

@zhouyx is there any information on what is expected in the consentMetadata as part of reject and accept actions?

[micajuine-ho]

@vyoungnyc We are still working on the consentMetadata field that can be sent via external iframe consent flow (accept/reject) and via checkConsentHref endpoint.

So far, we will be accepting 3 fields within consentMetadata:

• gdprApplies #28852
• additionalConsent #28707
• consentStringType #28028

Please let us know if you have any questions of concerns about these.

[vyoungnyc]

Also with the updates to webkit ITP and increasing difficulty for 3p cookies being set in an iframe, would it be possible to accept a field that persists for a cmp specific uuid for that particular user/consent string?

It would be much easier and dependable to be able to pull up the consentuuid and retrieve the up to date consent settings for that particular user without depending on cookies or working around itp.

[micajuine-ho]

@vyoungnyc Would something like CLIENT_ID, work in this case? You could get this value for the user and then have it sent to your backend to help calculate and update consentString.

[vyoungnyc]

CLIENT_ID could work if it can be included in the clientConfig passed to the checkConsentHref endpoint as well as the promptUISrc iframe. that way we can match it up on our backend and provide the up to date consentString

[zhouyx]

• CLIENT_ID is created under the amp-consent scope. Which means it's unique for the same user, not unique among different CMPs. In theory there's only one <amp-consent> allowed on the page, so there will only be one CMP. But in the case where a publisher switch to another CMP, the id is going to be reused. Does this work for you?

• While we could expand the support to clientConfig. CLIENT_ID is only supported in the promprUISrc and the checkConsentHref today. Example usage here
  https://github.com/ampproject/amphtml/blob/081a69f1f7a6416946f780bed31f01db70588d15/extensions/amp-consent/0.1/cmps.js#L31-L35

[micajuine-ho]

While we could expand the support to clientConfig. CLIENT_ID is only supported in the promprUISrc and the checkConsentHref today. Example usage here

Additionally, within checkConsentHref we send the current stored consentString and consentState which, along with the CLIENT_ID could help you determine new consentString and consentState. Does this work?

https://github.com/ampproject/amphtml/blob/081a69f1f7a6416946f780bed31f01db70588d15/extensions/amp-consent/0.1/amp-consent.js#L600-L610

[vyoungnyc]

1) That should be fine for us.

2) adding CLIENT_ID to the inline checkConsentHref and promptUISrc in the JSON data part of the CLIENT_ID does not get expanded and turned into an id when making the requests to checkConsentHref and promptUISrc. Is this macro only possible with the checked in CMP_CONFIG in cmp.js? we inline our config because our ccpa/tcfv1/tcfv2 have different endpoints.

Testing locally and also swapping the endpoints to prod doesnt expand the CLIENT_ID
<amp-consent id='consent' layout='nodisplay' > <script type="application/json"> { "consentRequired": "remote", "consentInstanceId": "cmp", "checkConsentHref": "http://localhost/consents/check?cid=CLIENT_ID", "promptUISrc": "http://localhost:8080/?cid=CLIENT_ID", "postPromptUI": "post-prompt-ui", "clientConfig": {} } </script> </amp-consent>

3) we would ideally want to be able to match it back to our own user consent id if our own id cannot be persisted because of expiry limits on localstorage or restrictions of cookies and not generate a new id on our end whenever they change states(as this would show more new users making consent actions than there actually is vs. existing user updating their consent status)

[zhouyx]

CLIENT_ID should work with inline checkConsentHref and promptUISrc. The change was merged recently and not in production yet.

[zhouyx]

Hello all! We're going to hold a breakout session about consent support in AMP during the online OpenJS World & OpenJs Collaborator Summit next week. (The AMP Contributor Summit has been merged into this summit this year)

We'll be covering updates on <amp-consent>, and are open to feedback and questions. Please register for the summit if you're interested! Registration for the online OpenJS World & OpenJS Collab Summit is now open!

Monday June 22 - OpenJS Collab Summit New Contributor Day
Tuesday June 23 and Wednesday June 24 - OpenJS World, bringing together the entire OpenJSF/JS/web community
Thursday June 25 and Friday June 26 - OpenJS Collab Summit Project Summit Days

Most of the AMP-community-specific talks and breakout sessions will be on Thursday & Friday, but everyone is welcomed/encouraged to attend all days to engage with the wider web/JS community.
When you register, make sure to indicate that you are part of the AMP community so we can get a sense of how many AMP people will be there.

The AMP talk & breakout schedule will be coming soon. :)

Thanks!

[zhouyx]

Here's the AMP talk & breakout schedule. https://docs.google.com/spreadsheets/d/1GY9qNrjxM6Ro-jFSGscL11sKQA46msRMdSrQMhTL8K0/edit#gid=461228868

[tla-sirdata]

Hi @zhouyx sorry I couldn't attend the talks... Is there any way to get details about the features and updates that have been announced ? Thanks

[zhouyx]

@tla-sirdata, there was some discussion around more granularly support to handle consent on the client side which is related to #26735.

[hrkhal]

@micajuine-ho Is the Immediate Lightbox change for promptUISrc iframes in production? If so how is it enabled?

We're keen to migrate to it for our TCF2 implementations.

[klipstein]

We would be interested whether amp-iframe or amp-video-iframe will receive the capability to read the TCFv2 string or whether we could help implementing this feature? Also see https://github.com/ampproject/amphtml/issues/28719.

[zhouyx]

@klipstein Thanks for opening the issue. It would be great if we could get your help implementing this feature! Let's track the work there.

[MaximePlancke]

Hello @micajuine-ho @zhouyx ,

We would like to know if all the attributes listed inside the consentMetadata object are all on production? (gdprApplies, additionalConsent and consentStringType)

Also, would it be possible to have some documentation about it? For instance how consentStringType works and its different values : https://github.com/ampproject/amphtml/pull/28028

[micajuine-ho]

@hrkhal I am working on the PR now (#29204). Hopefully we will get this out in prod soon.

@MaximePlancke additionalConsent & consentStringType are both in production, gdprApplies is still in beta (should be out next week).

Edit: Here is the PR with documentation on consentMetadata: #29228

[hrkhal]

@micajuine-ho / @zhouyx Can the character limit for stored consent information be increased significantly from the existing 1200 characters?

If the user "Allows All" non-TCF vendors then the generated additionalConsent string, as described here and using this vendor list, is around 3.5k characters long and passing it back into AMP from our external CMP as consentMetadata.additionalConsent throws the following error:

[CONSENT-STATE-MANAGER] Cannot store consent information which length exceeds 1200. Previous stored consentInfo will be cleared

Which, as expected, causes our consent information to not persist.

Additionally do you have an ETA for the light box ui?

[zhouyx]

Hi @hrkhal, we do realize the consent string requires up to 4k characters in size. But choose 1200 characters because it is a value that can fulfill more than 98% use cases, and is also within the 2k AMP Viewer storage limit.

Do you have an estimate on how often such error happens on your end? Asking because I couldn't located the same error in our error log the last time I looked at it. Thanks.

[Facens]

@zhouyx We actually have strings that are 8kb. With publisher restrictions in place, the string can get very large very quickly.

[zhouyx]

Thanks for the information.

We would like to support the consent string as much as we can. The size limit unfortunately is a restriction from the AMP viewer. When a cached AMP document is served in the AMP viewer in a cross origin iframe, it doesn't have access to the localStorage. Because of this AMP viewer supports a storage API which allows the AMP document served on xxx.cdn.ampproject.org to send storage value to the top level window via postMessage, and store the data to the top level domain. This means all cached AMP documents share the same localStorage space within the same AMP viewer. Some restrictions on the size must be applied otherwise the size limit will be reached very soon.

The feedback we got from the IAB team is that the 1.2k limit is going to fulfill most use cases (> 95%) which we find to be a good balance point between space restrictions and coverage.

As mentioned, the concern is about AMP documents served from the cache. We could remove the size restriction completely when the site is directly served from origin if you find the optimization to be worth it.

Thanks. Let me know what you think.

[hrkhal]

@zhouyx The issue (for us) isn't the length of the TCFv2 consent string but rather the length of Google’s Additional Consent Mode consent string that we'd generate for our users.

Across our brands a high majority of users grant us full consent which would generate the full length additionalConsent string, with every vendor ID, for all of those users. if we were to use use that in production AMP all of those users would be re-prompted every visit which is unacceptable and would more than likely reduce our consent rate in the long run.

While I understand the technical limitation it is far from ideal that we can't support a full length additionalConsent string even though AMP has defined a specific storage for it in consentMetaData.additionalConsent.

Given the re-prompting we can't feasibly support additionalConsent in all AMP documents.

Removing the 1.2k limit in non-viewer mode could be worth it. If we did progress with that optimization it should flagged that it is active in the clientInfo.

[Vasile-Peste]

As mentioned, the concern is about AMP documents served from the cache. We could remove the size restriction completely when the site is directly served from origin if you find the optimization to be worth it.

Hi @zhouyx, can you please elaborate more this part? Do you offer this option at this time? We would like to understand how to remove this limit: as @Facens said we actually have strings that are 8KB.

[zhouyx]

Thanks for the info. Didn't expect the addtionalConsent string to be so big.

Previously we decided to always apply the size limit because we figure it is best to match the behavior with or without viewer. As we get more request to remove the size limit, now I believe it is best practice for AMP to remove the restriction whenever it is not required. We need to clearly document the difference, and potentially provide a way for CMPs to easily test the behavior
when storage API is not used.

I agree with @hrkhal that the CMP needs to be informed of the running environment. We can test if the information can be accessed by location. ancestorOrigins or via the referral.

to @micajuine-ho

[zhouyx]

@Vasile-Peste
The option is not available at this time. We are working towards removing the restriction when the AMP doc is served directly from the publisher's origin. (When localStorage can be used)

[micajuine-ho]

Additionally do you have an ETA for the light box ui?

@hrkhal Lightbox consent PR has been merged.

The option is not available at this time. We are working towards removing the restriction when the AMP doc is served directly from the publisher's origin. (When localStorage can be used)

I created an issue to track the work for this project: #29824

[SyntaxNode]

Do RTC Vendors have access to macros which pull from the consent metadata?

It looks like CONSENT_STATE and CONSENT_STRING macros are defined in real-time-config-manager.js for RTC Vendors. Are there plans to add consent type and gdpr applies? .. or are those macros resolved elsewhere?

[zhouyx]

RTC vendors don't have that today. It's a good point, I think we should add the support.

@micajuine-ho could you propose a design? Thanks

[micajuine-ho]

Hi @zhouyx, can you please elaborate more this part? Do you offer this option at this time? We would like to understand how to remove this limit: as @Facens said we actually have strings that are 8KB.

Hi, we'd like to get some more context on what CMPs would do in each case, if the size restriction was lifted for documents that are not served from a cache.

Additionally, documents served from a cache through a viewer does not guarantee LocalStorage API support; thus consent information will not be persisted for those certain viewers.

[renebaudisch]

Hi everybody, I'm working for a 3P ad marketer and maintenance an amp-ad adapter and I right now do not see any consent informations inside an amp-ad iframe.

Also I strongly recommend using __tcfapiLocator as the official standard everyone refers to, to ensure adnetworks can be sure to get the needed consent information and not use nothing instead, so this means option 1 from the description.

Besides I do not understand the Con:
_Having CORS iframe running in the background can lead to performance and security concerns._
The hole amp page is designed by Cors-Frames, each ad frame is on a different origin as the mainframe and therefore has to regard CORS.

So option1 wouldn't have any cons, option2 has and will have cons.
So what's so bad on enabling that?

[zhouyx]

Thanks for the feedback. I can relate to the concern of lacking support to the TCF API __tcfapiLocator.

The current approach is selected because we think it's better for performance and user experience in a few ways

1. AMP can used the locally cached value to unblock content without waiting for the CMP iframe to load
2. AMP has better control of the UI flow. (No prompt will be allowed if there's locally cached consent)
3. The CMP iframe will be killed once consent has been collected. This is to prevent the iframe from running in the background and block the main thread.

I get your point that ads are already running in CORS origin iframe. Which we introduced the friendly iframe rendered AMP ad aiming to improve that.

We're happy to review the decision if situation change. But based on the feedback we gathered, ad networks and CMPs are OK with using the AMP provided APIs. So we chose option two which is a slightly more performant and user friendly option.

[renebaudisch]

Sorry, that doesn't convince me.

1. providing a method inside an iframe does not affect what happens in the main window, so you still could provide your own locally cached value to the iframes by the way the IAB has committed to.
2. isn't that the normal way every cmp should work? The one we have to use do the same.
3. as stated, to provide __tcfapiLocator like specified by IAB to the amp-ad frames you can still do what you want to in the mainframe.

To be clear I'm just talking about the specified accessibility of consent information within an iframe, and __tcfapiLocator shouldn't be bad for the performance since behind that method there still is amp javascript that regulates what it passes to the callback and what happens to collect the information.

So what I'm voting for is to register this mehtod as specified to ensure everyone get's consent the way it is specified in every environment, I'm not talking about the way of collecting it or passing it to this method at all.

So when you say

We're happy to review the decision if situation change

what exactly is the "situation" and what would be a change that "enables" a review of this topic?

[micajuine-ho]

So what I'm voting for is to register this mehtod as specified to ensure everyone get's consent the way it is specified in every environment, I'm not talking about the way of collecting it or passing it to this method at all.

Hi @renebaudisch would you mind expanding on this a little bit more?

[micajuine-ho]

@SyntaxNode I created an issue to track work for the consent metadata macros for RTC vendors here: #30006

[renebaudisch]

Sure, what I want to say is, that I want my code inside your amp-ad-frame to be able to do this:

https://github.com/InteractiveAdvertisingBureau/GDPR-Transparency-and-Consent-Framework/blob/master/TCFv2/IAB%20Tech%20Lab%20-%20CMP%20API%20v2.md#is-there-a-sample-iframe-script-call-to-the-cmp-api

This way I can consume consent provided by your cmp the same way I can on regular websites.
How your cmp before has collected data or the way it will gather the data to provide it still can be AMP specific.

Hope that explains my proposal.

[ahmedkaludi]

Hi,

I'm the lead developer for AMPforWP & WPQuads plugin. Our users are requesting TCFv2 support in our plugins and we really want to implement it in-order for them to stay up to date with the privacy policies.

Guys, Let's get this rolled out soon and have a solution for us.

[zhouyx]

@ahmedkaludi

Thanks for reaching out. AMP does provide TCFv2 support. https://github.com/ampproject/amphtml/blob/master/extensions/amp-consent/amp-consent.md#does-amp-support-the-iab-tcf

Not directly via the __tcfapi API because AMP doesn't allow CMP scripts directly running on the AMP doc, also because of how AMP stores values in the AMP viewer. However AMP provides a list of APIs for CMPs and ad vendors to pass TCFv2 information.
Vendor facing doc
CMP facing doc

Let us know if it work for you. And if not what's missing. Thanks!

[Facens]

@zhouyx the latest version of the TCF library manages to compress the 8kb string to around 1.7, so we could fit the 2k limit which is also the safe limit for URLs and which could be a good threshold for AMP, up from the 1.2kb that's in place now. Could this change be made? I remember this would be your upper limit, and I'd go for it as it would drastically reduce the edge cases.

[zhouyx]

@Facens Thanks for the information. Is the 1.7kb the upper limit for TC string now? Could you please refer me to the IAB doc. If it's going to stay like that, then I agree with we could revisit AMP's size limit.

[Facens]

@zhouyx the string has no upper limit and could grow, but since 2000 characters is what's generally considered to be the safe maximum threshold in other contexts (e.g. URLs and legacy webservers/browsers), in Tech Lab we're making all possible efforts to shrink down and stay within that limit. I was giving 1.7k as a reference because it's at the moment a size that is reached in a relatively low but significant number of cases and can be a deal breaker for some large publishers when approaching AMP. My suggestion would thus be that you increase the size all the way to 2000 chars which as you referred to previously is the max that you can get to without refactoring, particularly since there's Google's additional consent mode string to fit in too. Please let me know if we're aligned!

[vyoungnyc]

@zhouyx @micajuine-ho

Hey, we talked about using google's CLIENT_ID to sync up with our internal consent uuids. I seem to be running into a problem with CLIENT_ID being different when the amp page is hosted off the publisher website vs. off the google.com TLD.

e.g.
https://www.google.com/amp/s/www.publisher.com/amp/en-us/content-video-123
has a client id of something like: SPB8rlQkK8dFUnpCGjto31NL7P5g-j06ZJAn8tOoWv3G4oZdpw-deDoCn9YTiqhJ

but when you go to the publishers amp page of that same content in that same browser without clearing any cookies/local storage: www.publisher.com/amp/en-us/content-video-123
the client id is something like amp-nOpe-wXCUM1STTYOlYuSqg

i think the expected behavior would be that the client id's would be the same, so we can properly sync up the client id with our user consent uuid. but because it is different, the consent preferences of the user does not persist.

What is the best way to resolve this? because when amp pages are served off the google.com TLD, the cookies arent accessible when they go to a publishers page, so we are relying on google to provide a consistent CLIENT_ID

[vyoungnyc]

@zhouyx @micajuine-ho

Havent heard from you guys in awhile, should I create a new issue for the above?

[micajuine-ho]

@vyoungnyc Sorry about that. Yes, it would be appreciated if you could create a new issue, so it's easier to track/discuss. Thanks!

[alextcone]

@Facens wrote:

@zhouyx the string has no upper limit and could grow, but since 2000 characters is what's generally considered to be the safe maximum threshold in other contexts (e.g. URLs and legacy webservers/browsers), in Tech Lab we're making all possible efforts to shrink down and stay within that limit. I was giving 1.7k as a reference because it's at the moment a size that is reached in a relatively low but significant number of cases and can be a deal breaker for some large publishers when approaching AMP. My suggestion would thus be that you increase the size all the way to 2000 chars which as you referred to previously is the max that you can get to without refactoring, particularly since there's Google's additional consent mode string to fit in too. Please let me know if we're aligned!

Hi. I am the product leader for TCF at IAB Tech Lab. I'm getting a lot of questions about this AMP limitation on string size. Is it possible to make the change @Facens suggested in the quoted comment above?

[zhouyx]

@alextcone Thanks for reaching out

AMP has removed the consent string size limit when the Storage API is not used. That applies when the AMP page is served directly from the publisher's origin.

Regarding to the AMP viewer case, this is a known technical restriction on AMP. We have expanded the consent string size limit to as much as we can support today. I can reach out to the AMP viewer team on potentially expanding the size limit (maybe by another 50%), but that cannot solve the size limit issue either. I was told that the current size limit could fulfill 95% of all use cases. If that's still true, I am leaning towards keeping the size limit unchanged.

[micajuine-ho]

@SyntaxNode I created an issue to track work for the consent metadata macros for RTC vendors here: #30006

The PR has been merged.

[nicobatalla]

Hi @zhouyx . I'm creating an amp-consent solution for Quantcast CMP. I have pretty much all working by now, but I have a question. I saw that you can save a consentMetadata like this:

{
"consentStringType": {enum} [1: TCF V1, 2: TCF V2, 3: US Privacy String] (optional),
"gdprApplies": {boolean} (optional),
"additionalConsent": {string} (optional)
}

But how is that read from the amp-ad component? I'm getting the full TC string on initialConsentValue via window.context and the AMP consentState, but I can't seem to find the additionalConsent string anywhere, even though I'm setting it as consentMetadata via postMessage from my iframe.

[micajuine-ho]

Hi @nicobatalla, I assume since you're using window.context to read consentState, that you are trying to read the consentMetadata value from an amp-ad-3p-impl. If so, then here is the relevant issue #30786; we will be looking to expose the consentMetadata value through the window.context soon. Please let me know if this is not the case for you.

[renebaudisch]

@nicobatalla @zhouyx
Wouldn't this be obsolete when going towards IAB implementation as mentioned here #30385 which is intented to implement?

[micajuine-ho]

Wouldn't this be obsolete when going towards IAB implementation as mentioned here #30385 which is intended to implement?

My understanding is not all vendors/publishers will support an IAB implantations so it is good to accommodate these cases. I also don't think this change would be a large and will not have an impact on performance.

[renebaudisch]

Taking about advertising everybody SHOULD follow the IAB (International advertising bureau) and if not - you force everyone to go your way, so why not here too, rely on IAB and force everybody to use it?
Right now you/AMP are/is doing the same the other way round, I cannot get tc data as I'm use to, I have to go your idle way, that only exists on AMP.
But it's your baby not mine...

[nicobatalla]

@micajuine-ho @renebaudisch Thank you guys, I'm going to be following both issues.

[micajuine-ho]

@nicobatalla #30820 has been merged.

[tla-sirdata]

Hi @micajuine-ho do you have any update on the lightbox flow you were intended to implement ? Thanks !

[micajuine-ho]

Hi @tla-sirdata here is the PR (https://github.com/ampproject/amphtml/pull/29204), and the documentation can be found here.

[tla-sirdata]

@micajuine-ho oh i completely missed this out! Thanks

[micajuine-ho]

Hi all! FYI, we have gotten some asks to allow for CMPs to pass in a PurposeOne (boolean) signal to amp-consent. This signal, if found will give integrated vendors the option to interpret the signal and make a client-side decision based upon this, rather than sending the full TC String to their servers and making the determination server-side.

Here is the relevant issue. (https://github.com/ampproject/amphtml/issues/31144)

Please feel free questions or concerns to that thread :)

[micajuine-ho]

Hi all!

The purposeOne metadata field change has been merged (#31145). FYI in the future, ad vendors can chose to interpret this signal and potentially drop ad requests when purposeOne consent is not explicitly passed in.

Currently, passing in purposeOne consent will not have any affect on monetization, but including could help guard against a change like this occurring without notice.

[opencmp]

Hi!

As a reasonable solution, CMPs could add an attribute in the check endpoint's response to indicate to AMP that we want to display the UI while keeping the existing consent string.
On AMP side, a capping could be managed to prevent CMPs from displaying the UI too often.
This solution doesn't require a CORS iframe.

Has there been implemented a solution yet? I can't find anything in the docs except expireCache.

Is it possible to give more control to the CMP? Right now AMP is in control of the CMP. But in my opinion the CMP needs to be in control of when to show the popup. Only the CMP knows about changes in the vendorlist and in our case the CMP can be configured by our users to control its behaviour. But all that can't work in AMP because the iframe is not loaded.

A solution where the CMP always loads and takes control of AMP through APIs (show popup, close popup, update consent) would be very helpful. It should also be possible to store meta data with the consent string, i.e. the version of the vendor list or other information.