Sway: Nearest/smart scale_filter flickering back and forth linear/nearest

[luispabon]

• Sway Version:
  sway: ad189d27f97076c11e87ecd3b89859a49f58b031
  wlroots: 8681e4ab8a2d4c95abd34abf0e0eed3351d11bf0

• Debug Log:
  n/a

• Configuration File:

sway output eDP-1 pos 0 0 res 3840x2160 scale_filter nearest

• What happened

I've been testing out the recent work to xwayland scaling filters (https://github.com/swaywm/sway/pull/4727) for the last couple of weeks.

I have found one issue. Scenario:

• sway output eDP-1 pos 0 0 res 3840x2160 scale_filter nearest, also happens with smart)
• Open two xwayland apps side by side (for some reason it won't work if the other app is wayland native). It seems to happen more when two apps are side by side, not just one

When you click on app A (for instance pycharm), you can see app B changing its scaling from nearest to linear. Causes a very noticeable flickering. Sometimes it's only portions of that window that will go blurry, sometimes it's the whole screen. On Pycharm, you sometimes it's the currently selected line in unison with the cursor blinking.

It's impossible to screenshot. I'll look for a way to record a vid of it.

Before running this new feature, I had simply patched wlroots to always use nearest. As I understand, this patch works in conjunction with https://github.com/swaywm/wlroots/pull/1914/files (which is the line I was patching to use NEAREST) in the understanding that the default is linear anyway. When I patched that line (eg was always set explicitly) there was no flickering at any point.

• Stack Trace, if sway crashes:
  n/a

[emersion]

Maybe we're missing a call to set_scale_filter prior to render_texture. To fix this, I'd prefer to move the set_scale_filter call from inside render_texture so that we're sure not to miss new calls.

[luispabon]

https://www.youtube.com/watch?v=CBKAGahU8yw&feature=youtu.be --> make sure you're viewing the 4k stream on a 4k display, the flickering is a much harder to see at 1080p or less.

[luispabon]

@emersion do you mean calling set_scale_filter from render_texture? I tried it here and that didn't do it: https://github.com/swaywm/sway/commit/39a2e3998e47bc74a0095f15cd5bb5051fa2211b

[tokyovigilante]

Can 100% confirm this with two tiled Xwayland windows. Does not occur with just forcing GL_NEAREST in wlroots.

[RPigott]

Hey guys. I've borrowed some 4k hardware and been working on this for a bit but no solution yet. I'm using xterm as a test client. Using scale_filter nearest here's what I've observed:

1. I can't reproduce with only one xwayland client on screen.
2. If there is more than one xwayland client on screen, switching focus between two xwayland clients has a chance to give either the new or the old client "the badness".
3. Clients with "the badness" appear to alternate between linear and nearest on render. When left alone, for example, "the badness" xterm will alternate between linear and nearest every 5 seconds: whenever my swaybar updates/renders. When the cursor is moved (anywhere) on the output, "the badness" client will flicker very quickly as the cursor is rapidly redrawn.
4. It appears only one client can have "the badness" at a time. When a new client catches "the badness", the old one is cured.
5. Switching focus to a wayland client cures "the badness"
6. It is possible for the badness to affect only a portion of a client. After some testing, I have a hypothesis: parts of the client which are damaged after catching "the badness" are cured. e.g. while echo Hello, world! in a loop I can intentionally give my xterm "the badness" by shifting focus back and forth. When it catches "the badness", I can shake the cursor on the output and observe that only the "Hello world"s above the line it was on when catching "the badness" flicker. When "Hello world"s fill all lines and xterm scrolls to accommodate, "the badness" is cured.
7. Using swaymsg to switch between scale_filter linear and nearest can cause several clients to catch the badness at once. For clients that catch "the badness" upon switching to scale_filter linear, after "the badness" is cured by any of the above methods they do not flicker and do not catch "the badness" again.

I'm still not terribly familiar with wlroots & gles. If these observations seem interesting to you let me know and maybe I can figure out where to look.

[vesim987]

@emersion do you mean calling set_scale_filter from render_texture? I tried it here and that didn't do it: 39a2e39

This patch fixed the issue for me. Why this commit is not available in the commit tree anymore?

Edit: nvm, it doesn't fully fixed the issue, when I have two xwayland windows opened then the unfocused one is flickering.

[luispabon]

I realised as I was writing it that the patch wouldn't actually change anything, render_texture is called in 4 places, the two that were missing set_scale_filter were on things that don't require pixel scaling (window borders fi). Looks like something outside sway is resetting the filter to its linear default?

[emersion]

Logging the scale filter via glGetTexParam right before rendering in wlroots might help debugging this issue.

[RPigott]

Sounds good. I'll give it a shot.

[RPigott]

Logging works, thanks for the direction. Still don't know where the error is, but I think I've observed it in log form:

# Focused xterm has badness ?
2019-12-31 16:37:15 - [sway/desktop/render.c:237] Rendering toplevel: xterm (focused)
2019-12-31 16:37:15 - [sway/desktop/render.c:91] Set scale_filter GL_NEAREST
2019-12-31 16:37:15 - [render/gles2/renderer.c:159] GL_TEXTURE_MAG_FILTER: GL_LINEAR
2019-12-31 16:37:15 - [sway/desktop/render.c:237] Rendering toplevel: xterm (unfocused)
2019-12-31 16:37:15 - [sway/desktop/render.c:91] Set scale_filter GL_NEAREST
2019-12-31 16:37:15 - [render/gles2/renderer.c:159] GL_TEXTURE_MAG_FILTER: GL_NEAREST

# Unfocused xterm has badness ?
2019-12-31 16:37:24 - [sway/desktop/render.c:237] Rendering toplevel: xterm (unfocused)
2019-12-31 16:37:24 - [sway/desktop/render.c:91] Set scale_filter GL_NEAREST
2019-12-31 16:37:24 - [render/gles2/renderer.c:159] GL_TEXTURE_MAG_FILTER: GL_LINEAR
2019-12-31 16:37:24 - [sway/desktop/render.c:237] Rendering toplevel: xterm (focused)
2019-12-31 16:37:24 - [sway/desktop/render.c:91] Set scale_filter GL_NEAREST
2019-12-31 16:37:24 - [render/gles2/renderer.c:159] GL_TEXTURE_MAG_FILTER: GL_NEAREST

So far I've tried logging the toplevel title/focus in render_view_toplevels, the set scale_filter in set_scale_filter and the actual mag filter value right before rendering in wlroots render_texture_with_matrix.

It appears that the scale_filter does not apply sometimes, but I'm not sure why at the moment. I guess I'll throw more logging at it.

[emersion]

After more investigation, it seems like glBindTexture _sometimes_ resets the scaling filter. It's not easy to understand whether the scaling filter should be set to NEAREST or LINEAR in wlr_render_texture_with_matrix, because this function is called from wlroots itself too (e.g. from the Wayland and DRM backends when rendering the cursor).

https://github.com/swaywm/wlroots/pull/1992 is an attempt at making things more clear when debugging GL texture parameters. I also have a local patch which prints a stack trace when the filter is LINEAR.

[emersion]

Can you try https://github.com/swaywm/sway/pull/4903?

[luispabon]

Testing it now.

[luispabon]

@emersion as far as I can tell that fixes the issue completely :ok_hand:

[RPigott]

@emersion Thanks a million for investigating and fixing this! Patch working as intended here!