Mpv: Slow performance / frame drops for 4K 60fps h264 video playback on macOS (opengl-cb)

[struq]

mpv version and platform

mpv 6751b5b Copyright © 2000-2018 mpv/MPlayer/mplayer2 projects
 built on Thu Feb 15 16:38:34 EST 2018
ffmpeg library versions:
   libavutil       56.7.101
   libavcodec      58.11.101
   libavformat     58.9.100
   libswscale      5.0.101
   libavfilter     7.12.100
   libswresample   3.0.101
ffmpeg version: git-2018-02-15-8a8d0b3

macOS 10.13.4 17E150g

Reproduction steps

mpv --no-config --vo=opengl-cb --hwdec=videotoolbox [video file]

Expected behavior

Smooth playback at 60fps

Actual behavior

Lots of frame drops
Adding --video-sync=display-resample, or --opengl-pbo, or dropping --vo=opengl-cb, or any combination does not help.

Log file

http://ix.io/KFZ

Sample files

http://distribution.bbb3d.renderfarming.net/video/mp4/bbb_sunflower_2160p_60fps_normal.mp4

mpv was able to play 4k 60fps videos with --video-sync=display-resample back on an old version (couldn't remember exactly). Then it was broken by a later update. I thought this was supposed to be fixed by the new opengl-cb video output but it doesn't seem to be the case.

[Akemi]

in your log display-resample was deactivated on startup because it was too slow. a window size of 7680x4320 most likely doesn't help either and i would also suspect that an intel iris 6100 might be a bit slow. hwdec might do more bad than good.

start the video paused, start the playback manually after a bit and use 1:1 window size or fs.

yes the reported opengl related regressions have been fixed, that doesn't mean the horrible (opengl) performance of macOS will magically be gone or get better. i am also able to playback that file without problems, though with desktop hardware.

first we should probably measure your decoding speed. something like time mpv --no-audio --untimed --start=0 --end=60 --vo=null FILE (maybe try with hwdec too) and see what it outputs for real, user and sys times. if that's reasonable fast than there is most likely a rendering performance problem.

[struq]

Oh, that log was actually without --video-sync=display-resample, I did try both but only saved this log.

As per your suggestions, I tried the following configurations

1. --vo=opengl-cb --hwdec=videotoolbox --video-sync=display-resample --pause
   http://ix.io/KJS
2. --vo=opengl-cb --hwdec=videotoolbox --video-sync=display-resample --pause --window-scale=1
   http://ix.io/KJT
   This was actually creating the same 7680x4320 window as 1. So I tried the following
3. --vo=opengl-cb --hwdec=videotoolbox --video-sync=display-resample --pause --window-scale=1 --no-hidpi-window-scale
   http://ix.io/KJU
   The window size became 3840x2160.
4. --vo=opengl-cb --hwdec=videotoolbox --video-sync=display-resample --pause --window-scale=1 --fs
   http://ix.io/KJW
   full screen

Surprisingly, 1 and 2 were much better, but still had occasional stutters. 3 & 4 were terrible. In case 1 if I toggle fullscreen during playback, the frame drop starts.

I went back to 1 again without the extra --pause. But this time it wasn't as bad as in the original report, and appeared to be about the same as with --pause. The only reason I can think of is that an external monitor was disconnected. I could try that later.

I don't think the Iris 6100 is to blame. QuickTime, VLC, and Chrome's built-in player can all play without any visible skips, on either the laptop screen or the external 3840x2160 monitor. No problem with zooming, fullscreen, or seeking either.

time mpv --no-audio --untimed --start=0 --end=60 --vo=null
real    0m39.453s
user    2m21.063s
sys 0m1.812s

How to enable hwdec here? with --hwdec=videotoolbox it is still using 100% CPU and the time is about the same.

[valdotdev]

This just randomly happened to me last night. Every video I throw at it it stutters and drops frames. Works fine now after a restart. My sources were either 720p or 1080p

[AirPort]

This just randomly happened to me last night. Every video I throw at it it stutters and drops frames. Works fine now after a restart. My sources was either 720p or 1080p

I'm not 100% sure, but if it's what I think it is that's a system issue and has nothing to do (or nothing solvable by) mpv: the iGPU gets stuck to a low clock so graphical operations like scrolling, animations and of course video playback are serverely slowed down, and only a reboot can solve the issue.
In my experience it's rare but it happened to me a couple times in the past.

[Argon-]

@struq can you test again without hwdec? For me, on a 2015 MBP, it's not faster and in some times slower than software decoding. But what's even more important: it never uses less energy. I've checked that with Intel's Power Gadget. The used energy is always higher than with software decoding (or equal). Don't let yourself be fooled by lower CPU usage as hwdec shifts work onto the GPU where you don't see utilization as easily.

@AirPort @valdotdev there's currently a bug in High Sierra. After waking up from standby your GPU's clock/performance mode is decreased and won't get increased again until you restart. Afaik this has still not been fixed.

[struq]

I tested software decoding (--vo=opengl-cb --hwdec=no --video-sync=display-resample) and noted down the total power measured by Intel Power Gadget.

1. mpv (sw, fs) ~30W
2. mpv (hw, fs) ~8W, I noticed that GPU usage is low and is locked at the lowest frequency
3. mpv (sw, window 7680x4320) ~35W
4. mpv (hw, window 7680x4320) ~32W, GPU usage maxes out and frequency is higher

In terms of performance, my perception is 4>1>3>2. I'm not sure if the these make a good comparison about the decoding power because in all 4 cases the video can't be played at its full frame rate. So I also gave VLC a try (I'm not sure how to disable hwa only for decoding in Chrome). The power usage in fs is as follows. They don't seem to support a window larger than screen.
VLC (sw, fs) ~31W
VLC (hw, fs) ~29W
Sw decoding has some small skips. Hw decoding looks perfect and has an edge on power.

More on the external monitor:
Indeed the external monitor has an effect. I tried --vo=opengl-cb --hwdec=videotoolbox --video-sync=display-resample --pause in the following situations.

1. Laptop monitor only (2560x1600)
2. Laptop monitor (2560x1600) + external monitor (3840x2160), mpv window on laptop monitor
3. Laptop monitor (2560x1600) + external monitor (3840x2160), mpv window on external monitor

The playback performance is like 1>2>3. Fullscreen on external monitor also has the GPU frequency issue.

[valdotdev]

This has gotten pretty frequent for me as of late. Sometimes a few seconds after opening a video then it smoothes out by itself, again after a few seconds. Other times instantly and persistent. Granted, I too am using an external monitor (1080p only) but I'm not sure if it's related. But it was totally fine before the use of opengl-cb though so there's that.

[rien333]

As of yesterday, having rebuild ffmpeg and mpv, my Iris 6100 almost never reports any framedrops anymore, even with more fancy scalers applied (which work for the first time). Now, I must say that motion does seem less smooth with the scalers applied. When you guys talk about framedrops and stuttering/slow playback, is this just perceptual, or is this something indicated by mpv in the form of error messages or the display of Dropped: xx (i.e. indicating explicit framedrops)?

[ghost]

Dropped, mistimed, and delayed frames are going to count for smoothness. Also the "DS" value should be constant. You can see them on the status line, or the status overlay (i/I). Be aware that the status overlay can actually be slow to render and cause problems itself.

[valdotdev]

I usually have interpolation set on mine. In playback it was so jittery when in fullscreen. You could tell the difference when the interpolation kicks in when I go back to windowed mode. But as far as I can tell terminal outputs everything normally. I'm not sure if it's just my machine but I didn't have this performance problem a week ago. I'll probably disable scalers for the meantime.
Seems reverting the the old backend improves things but I'll stick with the new one for now.

[Akemi]

i doubt this is the problem, but i found a small issue that could impact smoothness of playback. fix can be found here (commit: don't render on displaylink thread, because it blocks). feel free to test and report back.

[valdotdev]

After comparing to the latest HEAD ver on mine, I definitely noticed less playback stuttering especially in fullscreen so I'm very thankful for that. Not sure if it completely fixes it though.

The horrible jittering/stuttering on master still happens very randomly and still can't pinpoint the cause. I'll see if these "random" slowdowns still happens after using your fix for a while.

EDIT: I just noticed a problem with the bundle in terms of playback performance (not sure it warrants opening another issue but tell me and I'll open one). I usually install mpv using homebrew brew install --HEAD --with-vapoursynth --with-bundle mpv unless I need to build manually, and most of the time the slowdowns are due to using the bundle (opening videos via Finder) as opposed to opening them via terminal.

[struq]

I tested version dfc9e033bc48ee686fb9e5d29870d733e5fa8de4 of the cocoa-cb_renderloop branch. There seems to be an improvement. Now there is no more stuttering with --vo=libmpv --hwdec=videotoolbox --video-sync=display-resample if the window scale is larger than 0.8, but not too large.

For example, --no-hidpi-window-scale --window-scale=1 or --fs on 3840x2160 display is smooth. GPU frequency is at 0.85GHz, utilization around 80%. --no-hidpi-window-scale --window-scale=0.801 is also good with 0.85GHz GPU frequency and 70% utilization. --no-hidpi-window-scale --window-scale=0.8 suddenly becomes stuttering. GPU frequency is at 0.45GHz with 30% utilization. Anything below 0.8 is similar, including 2560x1600 fs. For window scale > 1, there is an increasing number of dropped frames as I increase window scale. But the GPU frequency and utilization are also going higher so I assume this is limited by GPU performance and is normal.

---

Edit: Just tried latest version of master branch ed73ba89644fc66264f75e6dfdc4ff7bdf652739 and it's the same. But looks like dfc9e033bc48ee686fb9e5d29870d733e5fa8de4 is not in it?

[Akemi]

yes it isn't in master because i still need to test it properly.

basically what you are reporting sounds to me like shit power management. if the GPU isn't clocking properly to the load that is generated and consequential is too slow to render properly, there isn't much i can do about it.

what i noticed 3840*0.8 = 3072, which is also 2^10*3 = 3072. doesn't seem to be a coincidence and more like a hardcoded limit? maybe you could try with --geometry=3072x-2 and --geometry=3074x-2. if the latter works but the former doesn't the limit seems to be indeed at 3072.

[struq]

Indeed --geometry=3074x-2 works but --geometry=3072x-2 doesn't. I also noticed that if I start in 3074 and make the window smaller during the playback, GPU frequency would immediately drop. But if I increase the window size when the GPU frequency is low, it does not go up.

[Akemi]

i am wondering, did you ever try with iina? for testing it might be interesting if it performs better.

[Akemi]

another thing you could try is this branch. it enables multi-threaded execution for opengl. i didn't activate it since it didn't make any differences on my system, though maybe it helps on slower systems.

[struq]

Just tried iina 0.0.15. It is similar, but the cutoff window size is a bit smaller.

[ztarbugz]

Want to attend this discussion as I was running into same issue with the same file (Big Buck Bunny). I am owning a MacBook Pro 15 (2018). While checking some file for my configuration the Jellyfish example (4K HEVC, 30fps, 10-Bit, 200Mbps) was running without any framedrop on the Intel GPU. When using the Big Buck Bunny movie in 4K with 60fps, I get lots of framedrops. So for more testing I force the dedicated AMD GPU and I did not get any framedrop on this file, perfectly playing. I guess that the Intel GPU is not able to handle 4K 60fps. Will do some more testing, but IMHO it seems it is not an MPV issue.
For tests I also used vo=opengl-cb with Interpolation.

[Hrxn]

Yeah, 4K + 60fps + a high bitrate may simply be too much, especially with HEVC.

[ztarbugz]

Did more testing with other files. I could not find any technical information if the Intel GPU can handle 4K 60fps, but in my tests, no 4K 60fps was running fine (with AMD no problems).

[Akemi]

ffmpeg ticket https://trac.ffmpeg.org/ticket/8734#ticket

[Akemi]

this one was closed by ffmpeg as duplicate of #5609 for now. i made my point in the ticket that they are most likely not related.