Amphtml: Feature request: Add video support to <amp-img>

[pbakaus]

Describe the new feature or change to an existing feature you'd like to see

It would be super duper awesome if AMP supported Safari's syntax of sticking videos into an <img> tag, like so:

<amp-img src="gifsmustdie.webm" layout="intrinsic" width="2" height="1"></amp-img>

Describe alternatives you've considered

The alternative is using <amp-video>, removing controls, making it autoplay and loop. But I think the above is a much easier way to adopt as developer when replacing gifs, and gifs – let's be honest – must die.

[sparhami]

Just some notes from some reading I did (e.g. from https://calendar.perfplanet.com/2017/animated-gif-without-the-gif/)

• There are some performance benefits in Safari for going down the<img> route with a video compared to <video>
  
  
  
  • Avoids issues of <video> needing the main thread to be free before starting playback (no equivalent of decoding=async?)
  
  
  • Avoids extra round trips for checking content length / buffering, and instead simply requests the whole resource
    
    
    
    
    
    • The can be bad for performance if the resource is long, so this should only be used for gif-like things that are short
    
    
    
  
  
• Seems to be supported in Safari 11+
• The UX for these are akin to images (e.g. in Safari, long press leads to save image option)
• The Chromium bug for this is https://bugs.chromium.org/p/chromium/issues/detail?id=791658, which is a wontfix
  
  
  
  • Their perspective seems to be either use an animated webp or use a video tag (muted, autoplay, loop)
    
    
    
    
    
    • Webp unfortunately can be much larger
    
    
    
    • Webp may not have hw decoding support (maybe have it on some Androids?), which could drain battery a little more
    
    
    
    • <video> element doesn't get performance benefits from using <img> listed above
    
    
    
  
  

[sparhami]

@nainar Do you think this is something that might be worth pursuing? It seems like Chromium does not want to go down this path, but there would be some benefit for Safari. We could fallback to <video autoplay loop muted> for non-Safari browsers. One thing that is not clear to me is if there are any user activation requirements ever for muted autoplay (e.g. battery saving mode) and how the <img> UX differs from <video> in that case.

/cc @ampproject/wg-ui-and-a11y @ampproject/wg-performance

[pbakaus]

thanks for the context and thoughts @sparhami! If we do this, then we probably won't get the exact same perf benefits than the native implementation in Safari (at least in Chrome, where we can't map it directly to the native feature), but the main reason I want this is developer ergonomics, e.g. stopping gif adoption, and accelerating video adoption.

Chromium decided not to invest, but we could – after all, it's in our mission to improve the content ecosystem of the web on the framework level.

Also curious to hear @nainar's thoughts :)

[nainar]

I think this is definitely something worth pursuing.

• I would want to push Chrome to do this on their end though before doing it on our end so that we have the maximum possible gains.
• Also should this change be made in amp-anim instead since that is the component where GIF usage is higher?
• If we did create a <video> element as a fallback to <img> in Chrome - would devs who are styling say amp-img > img be affected?

[pbakaus]

• I would want to push Chrome to do this on their end though before doing it on our end so that we have the maximum possible gains.

That would be nice, but as they closed as wontfix I'm not sure how easy it is to revive. We can try though.. doesn't change the fact that because we're a framework, and we have a proxy for <img>, we can support it in the meantime, if we chose to.

• Also should this change be made in amp-anim instead since that is the component where GIF usage is higher?

Good point, yes! But I will say, I always found it strange that amp-anim exists. Why not runtime manage amp-img like amp-anim when we detect a moving image? Would really simplify the dev story.

• If we did create a <video> element as a fallback to <img> in Chrome - would devs who are styling say amp-img > img be affected?

yes, since we'd be sticking a video into the tag. But I think that might be acceptable. Still, a good point.

[nainar]

I will bring this up with Chrome members offline if we can engage with them that would be great otherwise happy to have this change. @sparhami how does that sound?

[newmuis]

@nainar or @sparhami can you let @ampproject/wg-stories know if this becomes a thing? Transforming this to a video behind the scenes might break stories, as stories expect to manage all media in the document.

[sparhami]

I will bring this up with Chrome members offline if we can engage with them that would be great otherwise happy to have this change. @sparhami how does that sound?

Sounds good.

Just to update the thread, potential concern we discussed offline was about whether or not the fallback route (using a video) would work in a reasonable way for things like battery saver or data saver mode. While iOS will not autoplay muted videos in battery saver mode, it seems like Android will play them. It isn't clear that this will be true going forward, but it seems like falling back to a video should be reasonable for now.

[newmuis]

@sparhami I believe that at least some versions of Android Chrome also do not autoplay videos without user interaction in data saver mode; I can try to dig up more details if desired.

[sparhami]

@sparhami I believe that at least some versions of Android Chrome also do not autoplay videos without user interaction in data saver mode; I can try to dig up more details if desired.

@newmuis That could be useful to know before we implement this. We tried on one version of Android, but it would be good to know if this might not work in some situations.

[gmajoulet]

For what it's worth, Android also has a system level option to disable video autoplay entirely. It's hard to know what % of people have it enabled though.