Amphtml: Design bucket: amp-list lazy rendering

[alanorozco]

Background

amp-list renders dynamically loaded JSON endpoints into an HTML tree. There are cases where the initial list tree can be server-side rendered, so an initial HTTP roundtrip has several drawbacks:

• creates unecessary lag between page paint and list paint
• may be harmful for SEO
• the no-content-resize invariant has to be enforced through partially fixed sizing, which is unnecessary when a full list is rendered statically, in which case the content could be clipped or its container oversized.

Goal

To provide a lazy rendering mode for the next version of amp-list that only fetches and renders dynamic data when state changes for:

• performance
• UX (no clipping/oversizing of static content per fixed sizing)
• SEO

Details

AMPHTML API

We can provide one of two different APIs in the form of static rendering or template-based rendering; each with its own drawbacks and benefits.

Static rendering

src is no longer required, but a bound [src] is. The component contains a subtree that matches the outer rendered structure of every amp-list: it should contain a child of type [role=list], whose children are all of type [role=listitem]. On build time, the initial list is left as-is, to be replaced once its bound [src] changes so its items get updated.

<amp-state id="myState">
  <script type="application/json">
  {
    "endpoint": "/my-search-api/"
  }
  </script>
</amp-state>

<!-- Example of how live search or autocomplete would be achieved -->
<input on="input-debounced: AMP.setState({
  myState: {endpoint: '/my-search-api/?q=' + event.value}
})">

<amp-list [src]="myState.endpoint">
  <div role="list">
    <div role="listitem">
      <a href="http://foo">My first static item.</a>
    </div>
    <div role="listitem">
      <a href="http://bar">My second static item.</a>
    </div>
  </div>

  <!-- N/A since rendering is O(0) -->
  <!-- <div placeholder>...</div> -->

  <!-- Every item is wrapped in a div[role=listitem], per amp-list's API -->
  <template type="amp-mustache">
    <a href="{{href}}">{{text}}</a>
  </template>
</amp-list>

This is the best-performing solution. It executes no initial-time JS-level rendering and time until display is only bound by the browser's rendering cycle.

The main drawback of this approach is that there may be structural discrepancies between the initial tree and dynamically rendered tree, caused by code duplication between the <template> and the static <div role=list>. In turn, this may cause styling issues that may be hard to track down by the author.

This drawback is minor. Ideally authors will source the static list and the template's contents from the same server-side renderer, which would eliminate this issue. But since we all make mistakes, shallowly different item structures could trigger a user-level warning (similar to how amp-bind warns per different default attribute value and bound attribute value) for ease of development.

A similar check could be performed in validation time instead, but since the structure of the template should match the static list only shallowly and maybe match only a subset of element properties, the complexity of such feature would be unknown. In either case, structural discrepancies can be prevented with good documentation and warnings are only a nice-to-have feature.

Template rendering

The initial data to be rendered is passed through an amp-state whose content's format should match the JSON endpoint response. On layout time, the component uses its <template> to render the static data.

<amp-state id="myState">
  <script type="application/json">
  {
    "endpoint": "/my-search-api/",
    "items": [
      {"href": "http://foo", "text": "My first static item."},
      {"href": "http://bar", "text": "My second static item."}
    ]
  }
  </script>
</amp-state>

<!-- Example of how live search or autocomplete would be achieved -->
<input on="input-debounced: AMP.setState({
  myState: {endpoint: '/my-search-api/?q=' + event.value}
})">

<amp-list src="myState.items"
          [src]="myState.endpoint">
  <!-- Initial <div role=list> is hydrated on layout time and filled with
       rendered static data. -->

  <!-- Applicable, since initial render takes time -->
  <div placeholder>...</div>

  <!-- Every item is wrapped in a div[role=list], per amp-list's API -->
  <template type="amp-mustache">
    <a href="{{href}}">{{text}}</a>
  </template>
</amp-list>

This approach is unnecessarily slow and wasteful. Its only advantage is that it forcibly ensures that static and dynamic lists render the same, since they're both sourced from the <template>.

Arguably, this provides the best DX since:

• items are always rendered with the same structure
• the developer cost of rendering static JSON that mimics the format of the endpoint is close to zero

However, UX > DX, and since this approach performs so poorly, it should be scrapped. It's only a partial solution as well:

• not great for performance, it prevents an initial roundtrip but still has to client-side render
• doesn't solve any of the SEO issues
• doesn't remove the no-content-resize invariant enforcements, unless we make amp-list render-blocking, which would be bad.

Allowing layout=container

The static rendering approach allows us to enable layout=container, since it maintains the no-content-resize invariant.

/cc @ampproject/wg-ui-and-a11y @aghassemi @nainar @cathyxz

[alanorozco]

@aghassemi I know we discussed some details offline about how the initially displayed static list should be sync'd with an amp-state. I filed this issue with the intent to brainstorm particulars, so feel free to drop in with said details!

[jpettitt]

What tells it to not render the initial list? The missing src="..." or something about the initial value selected for [src]?

Not sure if it's just an example or it has some magic meaning but having "/" as an initial value is probably not a good idea, developers will copy it and then wonder why they get errors when their home page doesn't parse as valid JSON. :-)

[alanorozco]

@jpettitt The missing unbound src is the signal for the dynamic list not to render on page load. I'm not sure I understand the endpoint issue, can you elaborate further?

[aghassemi]

related https://github.com/ampproject/amphtml/issues/15647 and https://github.com/ampproject/amphtml/issues/20641

Slotting for amp-list v2 design /to @cathyxz

I agree with the general assessment here, static rendering > client-side rendering of UserUX but not DevX. Technically this is already possible (<div> static list </div> <amp-list hidden [hidden]="state.listChanged..>...</amp-list> ), but I don't think it is a commonly used pattern.

[alanorozco]

Currently it can also be achieved through a static placeholder with the same tree as the rendered list, but that approach is kind of awkward and forces parse/render/repaint when it's unnecessary. There's also the whole no layout=container and resulting clipping/oversizing issues.

[jpettitt]

Re the end point issue: it's a documentation nit - if you put "/" somebody will copy it (the number of developers who start by copy pasting the example is large). If you put "https://example.com/my-list-endpoint" it's obvious that they need to replace it.

[alanorozco]

Gotcha, thanks for the feedback @jpettitt. Changed the endpoint prefix to /my-search-api/, hopefully that's clearer.

[sebastianbenz]

👍 I like this proposal! I've used Ali's approach (<div> static list </div> <amp-list hidden [hidden]="state.listChanged..>...</amp-list>) a few times, e.g. here. Being able to enable container layout makes things a lot easier.

[alankent]

Overall agree with the analysis, but some counter-arguments so they are on the table... If you want mass market adoption, you need to make it easy to do the right thing. Not everyone is sold, even today, that site performance and UX wins over DX because if it's too fragile, the next developer may break it and you regress.

Also, on the SEO front, not everyone agrees that having to process JavaScript to get the page contents is such a big issue. Too many sites need it these days. (I am not convinced of this personally, but smart people make the argument to me, so including here.)

One option is to make the AMP Mustache library available in JavaScript for developers to use. I thought I read some adjustments had been made (e.g. Mustache partials turned off). To guarantee identical behavior for SSR using the same template, it would be nice if the official AMP Mustache library was easy to use as part of the SSR engine. Otherwise there is a risk the SSR will be different for edge cases.

If I could have my cake and eat it too I would:

1. Support both static rendering and template rendering. Let the developer pick between them based on the tools they have at hand.
2. Extend the AMP Toolkit (optimizer) to convert template rendering to static rendering.

That way as a developer I simply drop the template in the AMP page and inject JSON if I have initial state. (I do not need to jump through hoops to share the template between the SSR engine and the AMP page.) Very simple DX, guaranteed in sync, and the optimized page returned from the origin is static rendered for ultimate performance and indexability.

[alanorozco]

Not everyone is sold, even today, that site performance and UX wins over DX because if it's too fragile

While I tend to agree, our design principles trump this. The point of AMP, from the beginning, was ensuring best possible UX, even if the developer had to go through additional hoops (eg AMPHTML validation).

Support both static rendering and template rendering. Let the developer pick between them based on the tools they have at hand.

This is possibly the choice we'll be going for, but I'm weary that AMP quality overall will suffer because people will just gravitate to the latter if it's "easier".

One big disadvantage of the template approach is also that layout=container cannot be allowed, which I think is important for the next version of amp-list/amp-render.

Extend the AMP Toolkit (optimizer) to convert template rendering to static rendering.

While I think this is a great idea and probably something we should do, virtually no-one uses the AMP optimizer. Not everyone uses node, so it would be nice to provide something compatible with other popular rendering languages like Python or PHP (despite my personal reservations about the latter 😉).

[cathyxz]

I think this the static rendering approach is a really neat way to get around the dynamic resizing problem (i.e. layout CONTAINER). For example, the shopping cart with checkout page, usually involves an <amp-list> with a variable number of items and a checkout form underneath. Due to the presence of a checkout form, it's impossible not to trigger the overflow condition if there are more items than the list height (overflow button on a shopping cart isn't great UX). But if you size the list so that the bottom of the list is outside the viewport, then you have an awkward UX where your payment button is artificially below the viewport with a bunch of whitespace in between. This is an extremely frustrating catch 22 situation, that rendering static items server-side could mitigate.

That does however, assume that developers have enough control over their endpoints and codebase to inject this data server-side. I don't know if that's a fair assumption to make--community feedback would be very helpful here. =)

What this does not solve (but more importantly I don't know if is compatible with) is the need to bind <amp-list>'s [src] to an <amp-state> that can be modified to user interactions. Specifically, client-side sorting and filtering. A few use cases:

1. I have a product search page, where each product has a like / favorite button that needs to show "favorited" / "not favorited" state according to user action (this data is initially fetched server-side, but I need to modify my like/unlike state client side without refetching all of my items .
2. I want to do all sorting and filtering client side.
3. I want to add / edit / remove items from a shopping cart (e.g. edit item quantity, remove items, etc. without having to ping the server on every action).

Most of this can be done by referencing an <amp-state> whose [src] is bindable, except when you have client-side filter/sort mixed with a server-side load-more, or when you basically need to preserve certain client-side state, but supplement it with data from the server. I don't currently know of a good solution for this problem, but how to make this compatible with load-more in general would be also be a good question to think about.

[samouri]

I believe almost every piece of this design bucket has been implemented in some form of or another.

• ✔️ Allowing for initial html content in the html instead.
• ✔️ Initializing from <amp-state>
• ✔️ Enabling layout=container

I believe the only unimplemented bit is that you'd ideally want the option to have the list _not refresh_ in case of initial html content, whereas right now we treat it as stale html and will update after an initial json fetch completes. My gut says that isn't a high priority, but if anyone thinks it is less split that out into its own targeted FR.

Closing