Amphtml: Allow `amp-list` to render from `amp-state` initially

[aghassemi]

See https://github.com/ampproject/amphtml/pull/15613#issuecomment-392893925 for some background.

We should allow amp-list to render from amp-state data on load. This can help with showing "stale" data until remote data is available. It would be nice to also prefetch the endpoint so when a user-interaction ( e.g. "refresh" button) switches the src from amp-state to real endpoint, we would have fetched the data already and swap would be quick.

/cc @choumx @ericlindley-g

[choumx]

FYI #12002 is a dupe. Also https://github.com/ampproject/amphtml/issues/12372#issuecomment-351829104 has some related discussion.

Basically we're wading deeper into client-side rendering -- might want to do some instrumentation on worst case scenarios.

[pancham348]

Hi, has this been implemented already ? If not, is there a workaround we could use for the time being ?

[aghassemi]

@pancham348 Note yet, you could potentially use <div placeholder> to render some slate items until amp-list loads, similar to https://ampbyexample.com/components/amp-list/#using-a-placeholder-animation but imagine placeholder were real items.

[pancham348]

ah that works! Thanks @aghassemi !

[demianrenzulli]

This FR, in combination with this bug, produces the following the following unintended effect:

This demo page is a simplified version of a shopping cart page:

• amp-list loads an initial a list of items, that should refresh when "shoppingCart.items" changes.
• Clicking the "delete" button updates another state object: "cartItem", to save the user selection, before showing a confirmation dialog (and triggering a form POST request if the user accepts).
• When the state object is updated, a global re-render is triggered and, since "shoppingCart" object hasn't been initialized, the list is refreshed with an empty value.

To avoid this behavior, e-commerce sites initialize the shoppingCart object with a client-side request, therefore, duplicating the request made by amp-list to load the initial list of items.

Wondering if there have been any updates on this FR or the bug, and if is there any workaround to avoid this.

// cc @morsssss

[alanorozco]

@aghassemi Should we close this in favor of static rendering per #20752? There's also nowhere to track synchronization between the static tree and an amp-state, so maybe we could change this issue to track that instead?

@demianrenzulli It's possible that synchronizing bound state from an initial static render (proposed feature) covers your use case. Leaving to @aghassemi to follow up.

[aghassemi]

Idea here was that all the data is managed to amp-state and src of amp-list is never changed. This means changing the src of amp-state to fetch fresh data.

<amp-state src="example.com/data" [src]="example.com/data?[filters] id=myState>
<amp-list src="amp-state://myState">

[danieltxok]

+1 to have this. It would make things much easier

[cathyxz]

We've slated this for a version 2 of amp-list and amp-render. If you have any sample code / use cases related to this FR, it would be very helpful to have them on this thread. =)

Thank you!

[morsssss]

Yay! @demianrenzulli do you have time to link to our code etc?

[demianrenzulli]

Correcting previous comments: the use case we were analyzing see demo, works well, without the need to render amp-list initially with amp-state:
Thanks to XHR Batching, amp-list and amp-state can be initialized with the same endpoint URL, and, only one request will be sent.
When any state object changes, a global re-render will be triggered, but since we have declared a [src] binding on amp-list, to the state object mentioned before, that would avoid refreshing the list as empty.

[morsssss]

By the way, we'd very much like it if AMP supported richer client-side rendering. I've heard that is a nongoal... is it still?

[alanorozco]

@morsssss It's definitely a goal. We're trying to strike a good balance between UX (ie fastest rendering possible) and DevX (ie easiest model to code for). The assumption on #20752 about this approach being scrapped doesn't consider possible third alternatives. We're not set on the design for amp-list/amp-render yet.

[dritter]

Our use case is slightly different from having the opportunity to display a better placeholder. We want to render the final state of the component based on a local state (that is currently only possible with a dummy request, just to trigger re-rendering the component).

To provide some more context:
We have several Microservices, and sometimes have the situation that we have the information for a component in one and the component itself lives in another one. Previously we transported that information/state across ESI-URLs, but that leads to a lot of cache buckets (which we want to to avoid).

Abstract example:
We have a header that contains a search input field. That input is the component I am talking about and lives in Microservice A.
Then we have a search results page (Microservice B) that embeds the header via ESI.
When the user searches for something, we send that request to the backend (so far, so good). That request comes back and we want to show the search query in the search box. Our previous approach resulted in a lot of cache buckets, as the search query is transported via ESI into Microservice A.

[aghassemi]

related: https://github.com/ampproject/amphtml/issues/20752

[alankent]

Example use case: The current amp-list causes page flicker as it refreshes the page after it is rendered. This is bad UX as the eyes are drawn the such flicker. You really want say a product list page to be pre-rendered server side so first display renders instantly, while still using an amp-list so you can change the sort order easily later.

You can do a trick like have two divs, one hidden one not, so the pre-rendered page is displayed first then hidden when the user changes sort order. But this adds developer maintenance overhead. Extra code is required, but worse two sections of the code have to stay in sync: you have to render the page server side plus create an amp-template that renders IDENTICAL markup (and keep both code in sync forever).

Using amp-state gets rid of the duplication. You inject the initial data into the amp-state, you render using amp-list and display the result on page load (zero duplicate code to render server side + via amp-template), then you can update sort orders and filtering easily. Ideally you update the amp-state, which triggers all amp-bind expressions and all amp-list templates on the page to update from a single API call.

[choumx]

Good timing, #23465 will help with the flicker. :)

[alankent]

Nice! But the flicker I was referring to is the delay in the amp-list going to the origin to do the API call to get the data to render the amp-list. I assume that is being looked at in amp-list v2...

[alankent]

Oh, also, I quite like the idea of being able to bind fields out of amp-state to a cookie. The cookie then sets the amp-state on page load, and updates to the amp-state field updates the cookie. Gives you state persistence across page boundaries (like a shopping cart, personalization preferences etc). That would then link in with amp-list using amp-state on page load as well. "Welcome back {username}".

[morsssss]

Is there any news on this, @cathyxz or maybe @nainar ? I've heard this requested by one of the largest makers of AMP sites in the U.S., as they want to render content without the delay caused by a server call. And I heard it again today at the AMP Roadshow in Warsaw, from another sophisticated AMP site developer.

[alankent]

One observation, https://amp.dev/documentation/examples/components/amp-list/?format=websites#rendering-amp-state documents how amp-list can be rendered from amp-state, but the documentation says that the amp-list is only rendered on state change. (The demo shows clicking a button updates AMP state which amp-list then renders.)

However, amp-list can fetch initial state using src="https://..." on page load from a URL, right? So on page load it will do the URL API call, return the content, render the template. But you cannot do this with AMP state. It treats "src=..." always as a URL.

It feels inconsistent that you can render URLs on page load but cannot render from AMP state on page load. But it also feels wrong inserting a bind expression into a src= attribute.

I understand there may be some race conditions to ensure the initial AMP state is initialized before bind expressions are evaluated on page load, but just wanted to suggest having a mode for: "trigger amp-list bind expressions on page load after AMP state has been initialized using [src]=... (and ignoring src=...)".

[choumx]

That's right. The original syntax proposal was in #12002.

<amp-list src="amp://foo.bar">
<amp-state id="foo">{ bar: ... }</amp-state>

This way we'd keep the semantics of [src] to "only applied during state change".

[morsssss]

That syntax strikes me as somewhat unnatural, since it looks to me like amp:// is a protocol like HTTP. If only we had some sort of character or string that implied "this is a state variable", like $ signals "this is a variable" in shell scripts or PHP.

I'm sure you've already considered this, but the other option would be to provide another attribute for this. I see how that could be messier.

In any case, I trust that you've thought about the options already. if people would like to prioritize this, however it's done, I think it would be a great feature!

[alankent]

I agree Ben, the syntax is unusual. Normally URLs are protocol://domainname - and foo.bar looks like a domain name but is not. It may also violate URL syntax rules if a full expression goes there.

I also think having to specify src="amp://foo.bar" and [src]="foo.bar" in this case is not intuitive. I want the same expression in both cases (that is, I really only want to specify [src]=).

I would consider other options like "if [src]= is bound to state and the amp-list has no default markup specified, evaluate it on page load". Or an extra attribute to trigger load on page load.

I think also potentially related is if AMP State was ever connected to cookies (mentioned above). E.g. I would love to fetch a cookie via AMP state and set a cookie by setting AMP state. Allow something like <amp-state id="blah" cookie="cookiename"> - this would allow state sharing across pages. (Alternatively, if an <amp-script> could be used to read or write cookies.) The relevant part to this request is it may be another reason to want to initialize the amp-list from state based on page load, as the cookie value can only be injected into the amp-list client side, and you want it on first page load.

[choumx]

Linking a related FR for amp-state error UI (#16011).

[morsssss]

We got another request for this feature from an experienced AMP developer for AndroidPolice in Warsaw last week!

Would we consider adding an attribute? Something like <amp-list src-var="foo.bar">? <amp-list src-state="foo.bar">?

If a new attribute makes <amp-list> too complex, I wonder about a convention like <amp-list src="#foo.bar"> - since at least #foo feels a bit like an HTML selector that finds the object with ID foo, which is what we're actually doing.

[morsssss]

Another way this request sometimes gets phrased is: "Allow JSON to load before the JS does." That way, when was ready to go, it could just use data that's already present.

[dritter]

@morsssss Nailed it!

Trying to clarify my abstract example: We have that situation all the time, where we render an amp-state in the backend (rendered by Service A), and need to pick up that state in an amp-list (rendered by Service B). It is this way, because the information needed to generate the state are only known in Service A, but not B.
Our options with this Scenario are either choose to create an endpoint to provide that state (which is kinda odd), or to create an dummy request in the amp-list to being able to trigger rendering from a local state (ugly workaround). We went with the latter one..

So for my part, it is not necessary to have a special syntax to access the local state. It would be enough to take the state (which should already be present) to populate the list.

[choumx]

It may also violate URL syntax rules if a full expression goes there.

In this case, limiting the initial expression to only JSON paths (e.g. foo.bar) might be desirable. This means the amp-list only needs to wait for the corresponding amp-state element to be initialized before beginning rendering.

OTOH, binding expressions (e.g. [text]="foo + bar") are not evaluated on page load. We could implement this, though it would mean an even slower initial render.

However, expressions do allow rendering from a multiple amp-state data sources instead of just one (e.g. [src]="foo.concat(bar)"). Is this desired or overkill?

That syntax strikes me as somewhat unnatural, since it looks to me like amp:// is a protocol like HTTP.

Correct, this would be a custom scheme/protocol. :) I think this is not too uncommon on the web. We can also drop the //.

<amp-list src="amp-state:foo.bar">

[alankent]

Good question on whether full expressions are overkill. For amp-list I would suspect yes. I can put the work into creating the JSON in the right format (using amp-script for complex logic). I am going to pull an array or similar out and pass that complex data structure to the amp-list.

I may even be happy with a single top level state name (no dots). As soon as you allow dots, do you allow [n] as well for arrays? The mustache template can pull apart the data structure.

`data-amp-src-linkage="foo"` ?

That might also make it easier to find the <amp-state id="foo"> to be loaded before evaluating the amp-list.

Would you support [src]= at the same time? I probably wouldn't. You specify a src (including using bind expression) or directly link it to a state member - one or the other.

(Side note, my understanding of URLs is even custom protocols are meant to follow the URI syntax. https://en.wikipedia.org/wiki/URL near the top documents the grammar. This is why file:/// uses three leading slashes - first two are the authority prefix, the authority is an empty string (localhost), then the next slash starts the path.)

[choumx]

I may even be happy with a single top level state name (no dots).

Only element IDs or JSON paths sound good.

As soon as you allow dots, do you allow [n] as well for arrays?

amp-list does support non-array data via the single-item attribute.

data-amp-src-linkage="foo" ?

I think I still prefer the plain src attribute since the semantics are clearer:

• src: The one and only data source.
• [src]: Only changes the value of src with setState. Otherwise, inert.

It also avoids combinatorial complexity introduced by a new attribute e.g. what happens when src and data-amp-src-linkage are both provided? What happens when there's a [data-amp-src-linkage] binding?

I think Ben's suggestion of a hash URL (e.g. src="#foo") could work too. We can discuss in design review, of course.

[choumx]

/to @samouri The first step to protocol adapters. :)

[alankent]

Just musing out loud. If dot is supported but not [n] it feels kind of inconsistent - you cannot specify a path into all structures. (JSON can have nested arrays etc.) If a single identifier, then using the src="#foo" is not bad. I cannot think of anything better.

[choumx]

I think we already support array indexing via dot syntax in JSON paths. E.g. object.array.1.foo.

Thanks for the discussion all.

[morsssss]

Reading said discussion... I do still like this syntax:

<amp-list src=#foo.bar">

But I also like the latest from @choumx :

<amp-list src="amp-state:foo.bar">

as its verbosity decreases ambiguity.

[samouri]

This feature landed in https://github.com/ampproject/amphtml/pull/27180.
Documentation is available at: https://amp.dev/documentation/components/amp-list/?format=websites#initialization-from-amp-state

🎉