Amphtml: FR: Introduce URI_ENCODE() for variable substitutions

[mattwomple]

Describe the new feature you'd like to see

It would be great if there existed a no-bind-necessary method of encoding URL components. For example: <amp-list src="https://example.com/json?myvar=URI_ENCODE(abc&def)"> would generate src="https://example.com/json?myvar=abc%26def".

A use case for this substitution would be <amp-list src=""> or <form action=""> rendered by <template type="amp-mustache">. Here is an example in psuedo-code:

<amp-list template="outer" src="https://example.com/first"></amp-list>
<template type="amp-mustache" id="outer">
  <amp-list template="inner" src="https://example.com/second?myvar=URI_ENCODE({{myvar}})"></amp-list>
</template>
<template type="amp-mustache" id="inner">{{{result}}}</template>

Describe alternatives you've considered

The method I use to handle this scenario right now is to provide some JSON members in both encoded and non-encoded form. For example:

{
  "myvar": "abc&def",
  "myvar_encoded": "abc%26def"
}

[Enriqe]

Pinging amp-list owners @ampproject/wg-ui-and-a11y, but feel free to redirect to a more appropriate working group.

[mattwomple]

I mentioned in slack #contributing that I'm only looking for "yeah, go ahead" or "nah, we don't want that" before I start working on implementing this feature.

[Enriqe]

@nainar do you know who would be able to help @mattwomple here?

[mattwomple]

Bump @ampproject/wg-ui-and-a11y . Still just looking for guidance on whether this feature would be accepted before I work on it. Thanks!

[zhouyx]

Thanks for the proposal. variables are encoded by default when they are used by <amp-pixel> or <amp-analytics>. The reason is that we need to encode variables to not break the outgoing request.
Is the proposed URI_ENCODE macro for <amp-list> only? Can we follow the same pattern and always decode variables?

[mattwomple]

Hi @zhouyx, thanks for the reply. I think the example I presented in my original post explains the most common use case: inserting mustache variables into URL query values. I understand that uri encoding is handled automatically for params attached to analytics beacons, and when using QUERY_PARAM for variable substitutions; this feature request serves a different purpose: to support uri encoding when generating src attribute values within templates. There may be other uses as well, but this is my primary goal.

[alanorozco]

There was a recent case where an API with parameterized URL tokens for amp-autocomplete had to be scrapped for email security requirements. Email requires srcs to be statically analyzable (see Caveats in #13457).

Per this rule, email already disallows mustache {{portions}} in src and related attributes, so that's cool.

However, the mechanism proposed here enables URL_ENCODE() at the amp-list level, so it can be used statically even without {{portions}}. I don't have enough context on this, so we need input from @ampproject/wg-amp4email to figure out whether A or B happens:

EDIT: nvm, figured it out :)

A. Per the existing static validation of srcs that Email already does, this type of token might already be rejected inadvertently. If this is the case, we can move forward and implement this in amp-list without changes to email. (EDIT: this _is_ the case, there's an allow-list for variable substitutions.)

B. If Email doesn't already reject URLs formed with these tokens, then Email's static analysis needs to understand them somehow, either by rejecting or unwrapping them like the runtime would.

/cc @choumx

So AFAICT there aren't blockers from the UI or email side. I'd like to see this being designed generically for applicable components, though. But I don't think that's a blocker.

[mattwomple]

I appreciate the context @alanorozco . Let me make sure I understand you correctly. AMP for Email disallows building src attribute values in mustache templates because those attributes must be statically defined. Your concern was that ENCODE_URI() could break src attributes in the same way, but you conclude that this isn't actually a concern because variable substitutions have to be whitelisted (e.g. using data-amp-replace="ENCODE_URI") and AMP for Email is designed to already disallow these variable substitutions. Is that correct?

I'd like to see this being designed generically for applicable components

Right, I never intended to tie this to amp-list or amp-form, these are just the examples that demonstrate my motivation for this feature request. My goal is to implement this very similarly to QUERY_PARAM. It would follow the same per-use opt-in rules required by other variable substitutions.

[alanorozco]

AMP for Email is designed to already disallow these variable substitutions. Is that correct?

Yep!

Right, I never intended to tie this to amp-list or amp-form, these are just the examples that demonstrate my motivation for this feature request.

Awesome, go for it.

[fstanis]

Hi all, just wanted to give some more information about email: currently, email clients block URL substitution by using <meta content="" name="amp-allowed-url-macros">. In our last WG meeting we discussed just making this the default in the runtime in the future.

In short, as long as this new substitution can be blocked via that meta tag, I see no concerns.

[zhouyx]

I still want to explore the possibility to have the extension code to decide whether to encode or not.

Also what are the extensions that you think will need this variable? If it's only used by template, should it be some macro provided by <amp-mustache> component?

My concerns behind are

1. Global variables are introduced to all extensions and built into v0.js. We want to limit expanding the variables list unless it will be used for all.
2. Personally I it will save publishers work if extensions can automatically encode urls. Like what we did for analytics requests.

[mattwomple]

I'm opting to drop this FR and associated PR since the only use case (in amp-mustache) would be better covered by a "data massaging" feature: #25719.