Html: Allow animations to be given a target framerate

[jakearchibald]

Problems:

We're seeing more devices that have screen refresh rates of 90, 120 and 144hz. Apple in particular are worried that allowing higher rendering rates by default will unnecessarily drain battery life. As such, on their 120hz devices, requestAnimationFrame is throttled to 60hz, whereas CSS animations run at 120hz. This doesn't really fit with the current spec, which expects CSS animations and requestAnimationFrame to run at the same rate.

On iOS native, a view can hint towards a particular framerate, and this defaults to 60hz, so developers must opt into higher rates.

It's currently difficult to lower the frame rate of an animation for stylistic reasons, eg 12fps for 'drawn' animation, or 24fps for film. With requestAnimationFrame you have to deliberately 'do nothing' for some callbacks, or try to wrap setInterval around requestAnimationFrame. CSS animations provide step(), but you can't mix this with other timing functions.

It's currently difficult to lower the frame rate to achieve a consistent rate across an animation. Eg, animation at 30fps rather than fluctuating between 30 & 60. With requestAnimationFrame you have to deliberately 'do nothing' for some callbacks. CSS animations don't provide a method here, although the browser can do it automatically.

It's difficult to synchronise animations that run on the compositor thread with animations that run on the main thread. If the main thread becomes busy, the main thread animation will have a different frame rate to the compositor-driven animation. This is usually ok, and maybe not something we should try to solve.

Goals:

• Allow developers to set a framerate for a set of animations.
• These animations may be of different types. Eg CSS, web animations, frame callbacks, animation worklets, SVG animations.
• The framerate may be a hint, which may be snapped to a multiple of the refresh rate. Eg, a desired rate of 30fps may become 25 on a 50hz screen.
• The framerate may not be a hint. 24fps may have been chosen to synchronise with a 24fps video.
• Different sets of animations may have different rates within the same document. A cartoon animation may run at 12fps, whereas other UI may be 60fps.
• The rate of animations may be changed during the animation, in reaction to an inconsistent rate.

Twitter thread: https://twitter.com/grorgwork/status/1185258811109433344

[jakearchibald]

Web Animation's DocumentTimeline feels like a good home for this.

It could be given properties for the target framerate, and whether it's a strict rate. These values could be changed throughout the life of the timeline.

const timeline = new DocumentTimeline();
timeline.targetFPS = 12;
timeline.strictFPS = true;

The timeline would provide a way to get a callback the next time its time updates. We could maybe explain requestAnimationFrame on top of this.

timeline.onFrame(() => { … });

// Maybe:
function requestAnimationFrame(cb) {
  document.timeline.onFrame(cb);
}

Developers could create additional timelines if they want groups of animations to have different rates.

const cartoonTimeline = new DocumentTimeline();
timeline.targetFPS = 12;
timeline.strictFPS = true;

const smoothTimeline = new DocumentTimeline();
timeline.targetFPS = 120;

document.timeline is the default timeline, so changing rate values here would cover all animations on the page, unless they were specifically put into a different timeline.

document.timeline.targetFPS = 90;

Browsers would be able to give default values to document.timeline, or even change the values unless they were explicitly set by a developer. This means a browser could limit animations to 60fps on a 120hz device, or even throttle to 30fps in battery saving modes.

This doesn't cater for the current iOS behaviour where CSS animations run at a different rate to rAF, but maybe that behaviour should change?

cc @birtles.

[AshleyScirra]

It would be good to also have a way of saying "use the display rate of the device" instead of having to hard-code an arbitrary FPS in that case.

[rniwa]

Not endorsing or opposing this idea but in the case of matching movie frame rate of 24fps, we’d also need to take motion blur into account:
https://www.red.com/red-101/shutter-angle-tutorial
https://beyondthetime.net/cinematic-motion-blur-180-rule/

[jakearchibald]

@AshleyScirra a target rate of Infinity should handle that.

[jakearchibald]

@rniwa there's some discussion over at https://github.com/w3c/csswg-drafts/issues/3837.

Edit: But I think the motion blur stuff is mostly separate. Some things benefit from blur, but if the style is 'cartoon', you don't want traditional motion blur there.

[birtles]

CSS animations provide step(), but you can't mix this with other timing functions.

This particular part can be done with GroupEffects (where you put the step timing function on the parent, and the other timing function on the child). That said, that only helps Web Animations (and presumably CSS animations in future) so it doesn't help with rAF or CSS transitions so I think this proposal makes sense. DocumentTimeline does feel like a suitable place for this.

A slightly related feature which has been come up and which may be worth considering at the same time is supporting frame-based animations, that is, animations where animation time progresses at a constant interval, regardless of the actual elapsed wallclock time between frames (i.e. prefer slow-down over dropping frames). For example, see "Duration based vs frame-count based animation" from:

• https://product.voxmedia.com/2013/11/25/5426880/polygon-feature-design-svg-animations-for-fun-and-profit

For that use case presumably the author would _not_ want the animation to run faster on devices that can animate above 60fps. This might suggest an API like targetFPS but rather than having strictFPS, have an enum value that has different modes?

[Yay295]

For that use case presumably the author would not want the animation to run faster on devices that can animate above 60fps. This might suggest an API like targetFPS but rather than having strictFPS, have an enum value that has different modes?

Something like timeline.fps instead of timeline.targetFPS and timeline.fpsMode instead of timeline.strictFPS, and then timeline.fpsMode could be one of strict, max, min?, etc.

For the syncronizing-with-video use-case there's this proposal, though it hasn't had any activity since July.

[jakearchibald]

@birtles

A slightly related feature which has been come up and which may be worth considering at the same time is supporting frame-based animations, that is, animations where animation time progresses at a constant interval, regardless of the actual elapsed wallclock time between frames

Interesting! I think something like that is complementary, but seperate. As in, it shouldn't be a goal here, except to ensure we don't prevent it from happening later. It feels like this feature would be a new unit for animation-duration. So instead of 3s it could be 200frames or whatever.

[jakearchibald]

There are a couple of things missing from my proposal:

• element.animate doesn't allow animations to be put on another timeline. So all animations will end up on document.timeline. You could solve this by adding an option to set the timeline, like you can when constructing an Animation. But maybe there's a reason this isn't already there? (@birtles?)
• There's no way to set the timeline for a CSS animation. There's already a proposal for animation-timeline, but it would need a way to access a developer-created timeline. That probably means we'd need some way to define a named timeline in CSS, similar to how @keyframes works.
• Opting into higher framerates requires JavaScript. Given that JS isn't required for animations, it seems sad that opt-in is JS only. I guess we'd also want a CSS way to influence the document timeline.

However, even without the above, we have a way to explain how browsers throttle the default timeline, and it provides a way for developers to opt into higher framerates.

[flackr]

You can approximate custom frame timing using animationiteration events: https://jsbin.com/cagehit/edit?js,output

I don't have a 120Hz iOS device to test whether the above demo allows JS 120Hz animations but I believe this also has the nice timing characteristics of firing at the same part of the lifecycle as requestAnimationFrame due animation events being fired when animations are updated: https://www.w3.org/TR/web-animations-1/#update-animations-and-send-events

[Yay295]

I have a 144Hz monitor. Your flicker() code was off by 1000 (seconds instead of ms), but otherwise it works. I added a FPS counter to the log: https://jsbin.com/bizuwuleja/edit?js,output

[birtles]

@jakearchibald

Interesting! I think something like that is complementary, but seperate. As in, it shouldn't be a goal here, except to ensure we don't prevent it from happening later.

Agreed. I think the frame-based animation feature boils down to two things:

a. A minimum interval in _wallclock time_ between frames (i.e. maximum frame rate). This is the part that overlaps with this issue.
b. Causing the reported _timeline time_ / _rAF time_ to increment by exactly the interval specified in (a). This is orthogonal.

I believe both those things should happen on the timeline, not as units on an animation.

• element.animate doesn't allow animations to be put on another timeline. So all animations will end up on document.timeline. You could solve this by adding an option to set the timeline, like you can when constructing an Animation. But maybe there's a reason this isn't already there? (@birtles?)

Correct. element.animate() was introduced as a shortcut to create an animation that (a) is playing, (b) has a KeyframeEffect as its target effect, and (c) is attached to the DocumentTimeline.

If you want to do anything different to that you either need to use the Animation() constructor or mutate the Animation returned from element.animate().

It's easy enough to add parameters to KeyframeAnimationOptions if it proves necessary, however. (If the parameter takes an arbitrary timeline, as opposed to a framerate, it might take a little more work because we need to work out how to handle inactive timelines in that case.)

• There's no way to set the timeline for a CSS animation. There's already a proposal for animation-timeline, but it would need a way to access a developer-created timeline. That probably means we'd need some way to define a named timeline in CSS, similar to how @keyframes works.

This is possible. We've talked about introducing @timeline in the past and I believe a version of the scroll-linked animations spec might have included it at one point.

[mdrejhon]

Founder of Blur Busters / Inventor of www.testufo.com here.

Lone Standout Situation (Apple): animations stuck at 60fps on 120Hz iPads

Earlier, W3C accepted my commit for fps=Hz requestAnimationFrame() in HTML 5.2 section 7.1.4.2. However, it appears WHATWG does not have a firm consensus about this behavior, creating a lone-standout problem (Apple 120Hz iDevices don't work properly with www.testufo.com ) when everything else works (Microsoft, Google, Samsung, OnePlus, HTC, Opera, FireFox, etc).

FAIL:

• Apple Safari on 120Hz iPads (WebKit)
• Old Microsoft Edge (old EdgeHTML)

PASS:

• Chrome (all; mobile and desktop)
• FireFox (all; mobile and desktop)
• Opera (all; mobile and desktop)
• Edge (all; desktop and tablet)
• Chromium (all popular forks)
• WebKit (most non-Apple forks)

Even the new Samsung 120 Hz phones (S11/S20) works perfectly in TestUFO (Google Analytics shows visits going green-120Hz from beta phones already). All other non-Apple high-Hz phones work.

I have submitted a WHATWG issue:

https://github.com/whatwg/meta/issues/158

Everything that animates need to consider high Hz, much like all APIs had to be updated to be retina-resolution-aware. CSS animations cannot serve all purposes, there's some very, very good reason (many unrelated to me or Blur Busters too!) that high-Hz support needs to be inclued in requestAnimationFrame() too.

Defacto Standard of High-Hz requestAnimationFrame()

Fixing this rAF() standardization hole at WHATWG is becoming urgent because of the upcoming boom of mainstream-brand 120 Hz smartphones (next iPhone and next Galaxy both support 120 Hz).

As a reminder for fps=Hz requestAnimationFrame()

Works -- All current versions of all desktop browsers
Works -- Razer 120 Hz phones
Works -- Pixel 90 Hz phones
Works -- OnePlus 90 Hz phones
Works -- Xiaomi 144 Hz phones
Works -- Samsung 120 Hz Galaxy (Google Analytics shows Samsung useragents green at 120Hz!)
DOES NOT WORK -- Apple 120 Hz iDevices

Need Solution Before 120 Hz Mainstream Boom

120 Hz becomes much more mainstream with upcoming 120 Hz iPhones and Galaxy launches this year. Even a temporary solution is urgently needed given the emerging tsunami of 120 Hz.

Before the 120 Hz iPhone arrives, maybe a temporary Safari-specific "vendor prefix" solution should be delivered as soon as possible (e.g. custom parameter like suggested) if Apple prefer to follow a differnt path to enabling 120fps with requestAnimationFrame()