Wlroots: Variable refresh rate support

Created on 3 Dec 2018 · 9Comments · Source: swaywm/wlroots

e.g. FreeSync

Upcoming DRM feature: https://lists.freedesktop.org/archives/dri-devel/2018-October/192874.html

11:13 <pq> Shibe, SardemFF7, emersion, DRM freesync support as currently designed will require compositors to implement support for it, and clients to opt-in to it. Mesa will intergrate some rules about which GL/Vulkan apps will automatically opt in. Additionally, it will only be used for fullscreen windows, never for compositor itself.
11:15 <emersion> what does it mean to "opt in"? is the blacklist a way to opt-out?
11:16 <emersion> for fullscreen windows, what should the compositor do?
11:18 <pq> emersion, yes, the Mesa black-list is to avoid opt-inning apps that should never get freesync enabled, that includes all X11 composite managers.
11:18 <emersion> oh, i see
11:19 <pq> emersion, the compositor needs to set the VRR DRM property to allow VRR, when it detects the window that opted in being on direct scanout.
11:19 <emersion> ah, so the "detect the window has opted in" part is the _VARIABLE_REFRESH X11 property?
11:20 <pq> the net effet of all this is that all hardware-rendering GL/Vulkan apps that show fullscreen will likely get VRR enabled automatically, without any changes to the app.
11:20 <pq> emersion, yes
11:20 <emersion> okay, so we'd also need a protocol for Wayland, right?
11:21 <pq> yes
11:21 <emersion> (used by mesa)
11:21 <pq> correct, AFAIU
11:21 <ascent12> When you say "never for the compositor itself", are you stictly talking about x11 compositors or wayland ones too?
11:21 <emersion> i think it's just for X11 ones
11:22 <pq> ascent12, the assumption is that all compositors, both Wayland and X11, have too unpredictable timings to be used with VRR. Wildly changing timings may cause "luminance flicker" on the VRR-supporting (poor) monitors.
11:22 <emersion> wayland compositors won't enable VRR on the DRM, so they should be fine
11:22 <emersion> *DRM side
11:23 <pq> yeah, a Wayland compositor will just know to not shoot itself in the foot, while X11 compositors are X11 clients to they depend on the blacklist.
11:23 <pq> *so they
11:25 <Deuxis> someone needs to make an API / framework version of http://www.toodarkpark.org/computers/humor/shoot-self-in-foot.html
11:25 <pq> I was very disappointed when I learnt that VRR monitor hardware assumes a stable refresh-rate and may cause issues otherwise. It's not a problem for all VRR monitors, but there is not way to know without a database that lists the identity of monitor models(?) that have a problem.
11:25 <Deuxis> so that we can make a wayland example
11:26 <Deuxis> wl_foot_shoot_foot(self, foot, callback)
11:27 <ascent12> It would be nice if the compositor couldv'e just dropped the frame rate as low as it can go when nothing is happening, so it can save power.
11:27 <Deuxis> that's the official recommendation
11:28 <ascent12> Maybe compositors could make vrr a toggable option so people with good monitors can enable it.
11:28 <pq> ascent12, indeed. To implement that with VRR, the compositor needs to keep flipping with VRR on even if nothing is happening.
11:28 <emersion> maybe it saves more power to just disable VRR in this case?
11:29 <pq> emersion, I don't know of the totals, but scanout does consume power and slowing it down would reduce power usage.
11:29 <pq> is it a net win or lose might even depend on the particular machine
11:29 <emersion> i assume there's no clear way to compute "as low as it can go"
11:30 <emersion> hmm
11:31 <pq> one could figure out by trying, but there was also discussion about exposing the maximum front porch extension supported
11:32 <pq> I suppose theoretically a compositor doesn't really need to keep on flipping, it can just enable VRR and stop flipping, so the driver will always time out and use the lowest possible rate
11:32 <pq> the problem is resuming updates which might then cause luminance flicker
11:33 <pq> so a knob for users could be a possiblity
11:37 <pq> (no, I have no idea how they deal with games that have problems maintaining a stable framerate - I guess they'll just flicker on bad monitors)
11:41  * SardemFF7 guess he has a not-bad monitor
11:42 <pq> btw. I never understood how wildly changing timings could cause luminance changes
11:42 <MrCooper> note that reduced refresh rate also affects HW cursor movement
11:43 <pq> MrCooper, but input events cause a compositor to resume flipping, so it'd hit max refresh rate ASAP. But if a compositor uses a slew rate to avoid luminance flicker, yeah, that could be visible.
11:44 <MrCooper> why would the compositor flip if only the cursor moves?
11:44 <pq> because that's how you move the cursor: atomic commit
11:45 <MrCooper> that won't flip if only the cursor plane moved, will it?
11:45 <pq> does that actually matter?
11:45 <MrCooper> I assume only a flip of the primary plane ends vertical blank before the timeout
11:46 <MrCooper> or any non-cursor plane maybe
11:46 <pq> why would you assume that?
11:46 <pq> at least Weston will fire up an atomic commit just like any other screen update and expect it follows the same timings and behaviour as flipping updates
11:47 <pq> Weston may also use the cursor plane for arbitrary windows, not just cursors
11:47 <MrCooper> because the cursor plane maps to the same old HW cursor registers, which AFAIK don't affect scanout timing
11:47 <pq> that sounds like what the legacy KMS API would do
11:49 <jadahl> pq: so a protocol is just a "opt-in-this-surface" that then may get vrr if it ends up going fullscreen
11:49 <pq> jadahl, yes, AFAIU
11:51 <pq> jadahl, if it ends up on fullscreen composite-bypass even, the Xorg implementation implicitly requires that
11:51 <SardemFF7> would that be ok as just an xdg_surface property?
11:52 <pq> SardemFF7, no, EGL implementations do not have access to any xdg protocol objects.
11:52 <SardemFF7> true
11:53 <pq> it needs to be something an EGL implementation can use without changes to apps
11:53 <MrCooper> pq: conversely, if the HW cursor did affect VRR, the refresh rate would likely be dominated by the cursor rather than the application's rendering during cursor movement?
11:54 <pq> MrCooper, right. I didn't think of games that actually use the window system cursor.
11:59 <SardemFF7> for these, does it matter? if they use the system cursor, VRR or not, they know the cursor will always be “ahead”
12:00 <MrCooper> not sure what you mean, my point was that the compositor using VRR to reduce refresh rate might result in choppy cursor movement
12:01 <emersion> well, as pq said, if the compositor correctly pageflips when moving the cursor it should be fine
12:01 <emersion> (assuming atomic)
12:02 <SardemFF7> didn’t pq said that the compositor would hit full rate quickly in this case?
12:02 <SardemFF7> also, VRR monitors cannot go /that/ low anyway
12:02 <MrCooper> that's based on the assumption that an atomic commit which only affects the cursor plane will end vertical blank before the timeout
12:03 <MrCooper> SardemFF7: users complain about 30 fps cursor movement
12:04 <SardemFF7> 30fps != 30Hz rate
12:05 <MrCooper> k... 30 Hz cursor movement then
12:05 <pq> MrCooper is correct too, we differ in assumptions
12:06 <MrCooper> I also pointed out that if the above assumption was true, HW cursor movement would result in the maximum refresh rate with VRR
12:06 <emersion> ugh, is the driver allowed to not vblank after userspace requested a pageflip?
12:06 <pq> emersion, a cursor position update is not a pageflip.
12:07 <emersion> well, if you commit with DRM_MODE_PAGE_FLIP_EVENT…
12:07 <emersion> (wlroots does, is this a good thing to do?)
12:08 <pq> weston does too
12:10 <pq> I've also seen talk about allowing fast cursor updates with atomic KMS, but I don't think that landed yet, did it?
12:10 <pq> that would be about allowing cursor position updates even if a flip was already queued, IIRC
12:12 <pq> MrCooper, maybe it would be better to have cursor position updates in a usual PAGE_FLIP_EVENT atomic commit to act as if the commit always flipped as well, and then use the new thing to do fast cursor position updates that don't honour the refresh cycle timings.
12:14 <MrCooper> maybe
12:15 <SardemFF7> I fave a question about the fps/Hz thing (no VRR case here): do I understand correctly that you can get 30fps updates on a 60Hz monitor, but that means each frame is shown twice ? Now on VRR monitor thas can do 40Hz to 60Hz, with no load and fast rendering, you cannot get less than 40fps cursor movement?
12:18 <Deuxis> AFAIU you can get any arbitrary fps on any arbitrary Hz monitor, as the monitor is dumb as bricks and will show whatever it finds in the buffer
12:18 <Deuxis> and it's not even about showing a frame twice, it's showing a frame for twice as long
12:18 <pq> SardemFF7, you can always got down to zero fps :-) It's just that the kernel DRM driver will keep on scanning out the same if the compositor doesn't update anything.
12:18 <Deuxis> there's just no update
12:18 <Deuxis> oh
12:19 <pq> SardemFF7, when the compositor is vsync'd, you can only do integer fractions of the nominal refresh rate: half, third, and so, or maybe you jitter and update on two vblanks of three.
12:21 <pq> SardemFF7, but you cannot achieved vsync'd fps that is between a half and a full refresh rate of the monitor without VRR.
12:21 <SardemFF7> hum, doesn’t help much… maybe with a more low-level question: do I understand correctly that if the monitor is set at e.g. 60Hz, it will always show 60 images of whatever is available on the fixed timing?
12:22 <SardemFF7> or do you mean it can show only 30 images (or 15) for real?
12:22 <pq> SardemFF7, what VRR does is that it allows the page flip to "arrive later" that the deadline without VRR, up to some maximum time, after which the hardware or driver will automatically start a new scanout cycle regradless but with old content.
12:22 <pq> *than the deadline
12:23 <MrCooper> the timing is always determined by the GPU, not the monitor; the only difference between VRR and not is the length of vertical blank being flexible vs fixed
12:25 <pq> SardemFF7, fixed refresh rate means the display chip in your graphics card will scan out the image always at that rate, continuously. (There are exceptions, where the driver actually can transparently switch from 60 to 30 Hz transparently, for instance.)
12:27 <pq> VRR does not change the "continuously" part, but VRR allows the new cycle to start either at the same time it would have with fixed refresh rate or some time later, up to some maximum timeout when it will start anyway.
12:27 <pq> the role of the monitor hardware here is to cope with the delayed start of the pixel data transmission for each monitor refresh
12:28 <pq> normal monitors can't do that, hence VRR monitors are a thing
12:29 <pq> neither is to be confused with a yet another monitor hardware type and supports updating contents on-demand instead of a continuous scanout cycle
12:29 <pq> *type that supports
12:32 <pq> SardemFF7, if you look at a diagram of VGA timings, those are for fixed refresh rate. There is a period called "front porch". The only change that VRR makes is that it allows front porch to be extended up to some maximum value which defines the lower vertical frequency limit of a VRR monitor.
12:33 <SardemFF7> I /think/ I get it
12:34 <pq> the lower vertical frequency limit is not the lowest fps one can display though, you can go even lower but that means that you skip a hardware refresh cycle completely
12:34 <pq> by skip I mean you show old contents
12:35 <SardemFF7> now back to the cursor movement speed, on a system with no load, fast rendering (or basically a fixed image), can I get a cursor movement slower than my monitor lower rate (35Hz for mine) or fixed rate (like 60Hz for the basic monitor)?
12:36 <pq> of course you can, by moving slower :-) That is, not moving on every monitor refresh.
12:36 <SardemFF7> sorry, let’s say my mouse is fast and I move it fast because I want to make sure I see the full potential of it :-)
12:37 <pq> so you are asking if your cursor might somehow be forced to update slower that you expect?
12:37 <pq> that depends on what the driver does and where my and MrCooper's assumptions differed
12:39 <pq> if there is nothing else updating on the display that the cursor position, you might get the fixed/highest rate or the lowest (VRR) rate of your monitor, depending on how the DRM driver does things
12:39 <pq> *than
12:41 <SardemFF7> my point is, you cannot complain about 30fps mouse movement if the compositor actually send more than 30 updates per second? (and the DRM thing is not ignoring it because it’s the cursor, which is your assumption?)
12:42 <pq> SardemFF7, the compositor is sending cursor position updates at, say, 1000 Hz if that's your mouse poll rate. Given that, you might get cursor position update on screen at either 60 or 35 Hz if that is your VRR monitor frequency range.
12:43 <SardemFF7> if my monitor can do 35Hz-144Hz, why would I get 35Hz? I would expect to at least going up to 144Hz in a matter of, I don’t know, less than a second?
12:44 <pq> SardemFF7, if cursor position update does not count as a pageflip, then there is nothing that would count as a pageflip, so the rate would never go up from the minimum.
12:45 <SardemFF7> that’s where assumptions differ, right?
12:45 <pq> yes
12:45 <SardemFF7> \o/ I understand
12:49 <MrCooper> thanks pq for explaining it better than I could have
12:50 <pq> haha, cheers :-)
12:52 <SardemFF7> are these assumptions something we can check somehow? it seems a bit bad that we don’t know what going on?
12:54 <pq> well, theoretically, if you record a timeline with Weston where nothing happens except the cursor moves, you should be able to compare the timings to when something else does update every refresh.
12:55 <pq> if pure cursor updates receive their page flip events too early, then MrCooper's assumption would be correct.
12:56 <pq> I have a feeling that intel and AMD might differ

enhancement backendrm

Source

emersion

👍12 ❤4

Most helpful comment

Wayland issue: https://gitlab.freedesktop.org/wayland/wayland/issues/84#note_136147

emersion on 30 Apr 2019

👍5

All 9 comments

Merged upstream in the kernel: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=bb47de736661887f37b5380f68a3faedf077aed0

emersion on 9 Jan 2019

lahvuun investigated and we concluded it's not possible to use VRR to reduce the framerate when idle to save battery. Next steps are enabling VRR support for fullscreen clients, by driving pageflips with client commits.

emersion on 17 Jan 2019

If I may ask, what was the reason behind it not being usable to reduce framerate when idle? It would otherwise have been a very interesting thing to do.

kennylevinsen on 9 Mar 2019

It results in flickers and glitches IIRC.

emersion on 9 Mar 2019

My understanding of this is incomplete, and I have never had the occasion to sit in front of a VRR monitor to test my theories. I also don't have a full understanding of this, but I'll write the following, hoping it can help (this is simplified to make it short and to the point).

But, for me there are two big categories of monitors that can be distinguished:

Full persistence displays
Low persistence displays

Let's start with the first one, which (nowadays) is the most common type of display. Each frame sent to the monitor is displayed one, and stays there until another one is displayed. Simple, basic. There might be some jitter if one image stays a lot longer than the other. Actually, there always is some kind of ~~jitter~~ "jagginess", since movement is sampled and held (traditionally for ~16ms or more). Blur is also very necessary on fast-moving objects (even more so as the refresh rate is lower), to give the impression of fast movement. Good luck trying to follow, say, the edge of a baseball bat while it's swinging.

Then, on to the second one. You likely stood in front of one if you ever saw a CRT monitor. Here, a beam refreshes pixels one at a time: only a couple pixels are lit up at any given time. CRTs did this at a constant rate. Movement can be interpolated in-between frames to give an analog, real-time image of what happens in-game (known as "racing the beam" ). When tracking moving objects, your eyes are not moving against a static background, thus less perceived motion blur.
Unfortunately, LCD panels (and similar tech) doesn't work like that, so instead, on high-end monitors the backlight is either scanning the image, or strobbed, while the panel displays a static image. That technology has a few marketing names, and goes with the _LightBoost_ moniker in nVidia circles. I just call this a low persistence display.

(this is now my own speculation and understanding)

Now, you might see where this is going: if we only illuminate the pixels when there's a new image, the luminosity is the sliding average of the backlight intensity times the refresh rate (for a given intensity, the more you refresh, the brighter the screen). So we need to compute beforehand how bright the backlight needs to be for the next frame. _I don't think there's a protocol for specifying this, unfortunately._ So, what could we do? Look at last frame's timing and extrapolate, I guess? Thus VRR can induce variable luminosity on low-persistence displays, when the refresh rate changes too quickly. I fully expect each brand to have its own algorithm, so it likely cannot be easily compensated.

As for how the monitors deal with framerate drops in games, it's probably variable as well. I guess some monitors could choose to maintain luminosity at the cost of some jitter, or the other way around. They probably filter the framerate to eliminate outliers from the calculation.

Takeaways:

Low-persistence (lightboost) displays can have luminance issues if framerate is too volatile
Monitors with luminance issues are likely the higher-end ones, counterintuitively
I wish there was a protocol to send predicted next frame time to the monitor (maybe even fall back to full persistence if unknown?), but I doubt there is.
VRR for compositors is probably doable, either simply (full persistence), or with a slew rate (low persistence). There might be a protocol to disable backlight strobing on some models? Who knows?
I'm waiting for Wayland support before buying one of these beasts :smile:

Supplementary material (there are tons of others, I'll let you exercise your search-fu :wink: ):

Strobed vs non-strobed video: https://www.youtube.com/watch?v=f0jlXUpG32Y
Another one: https://www.youtube.com/watch?v=v8BbVBrCVW4

I also didn't touch on ghosting, etc. which is out of scope for this...

MayeulC on 15 Mar 2019

👍1

How are other OSes handling this? I.e. is it the job of the compositor to set luminance based on the predicted frame time? Should compositor decide that based on whether the monitor supports it or not, and how would compositor get that information?