Handbrake: Support for GPU encoding on macOS - Enhancement.

[IComplainInComments]

Hello everyone, Apple has GPU en/decoding API's for macOS that utilizes Intel QuickSync along with other GPUs as well in macOS, I was wondering if HandBrake would be willing to commit these changes to the source to allow GPU encoding to be done?

https://developer.apple.com/library/content/technotes/tn2267/_index.html

Thanks!

[sr55]

We've already got a rough outline of a patch to add this but are debating whether we should. Unfortunately Apple has made an API that's one size fits all, so it cripples any app trying to use the true capabilities of the hardware. It's Apples way or nothing. So adding it leads us to the situation where you have great controls in windows, but 0 control under Mac and only basic Average bitrate encoding.

Also, it wouldn't be GPU encoding. It'd be ASIC's They are not the same thing. QuickSync isn't a GPU encoder. It's dedicated hardware which is much superior as it's single function.

[IComplainInComments]

Well the Apple lockdown 0 control thing is basically normal when it comes to Apple. As it would go through CoreVideo, and if I remember correctly, it would Match settings that are set by the Program anyway, as if I remember correctly. Apple uses the GPU to offload and assist, not necessarily put ALL encoding tasks on the GPU.

And not to sound sharp, but I know its an ISP on the GPU die that allows for QuickSync ;)

Either way I think it would be heavily beneficial even if there isn't that much of a control, as again CoreVideo would match settings and only offload tasks that it sees fit, which granted does remove some control from the user, but would greatly add performance to the systems in general. Just having the option would be wonderful.

EDIT: The offloading is an OpeCL module along with the software being able to see the SIP for NVIDIA/Intel chips.

[Djfe]

What am I missing?
you're talking about encoding but link to an apple document about a video decoder ^^

[galad87]

He linked some outdated documentation.
If you want to try QuickSync thru video toolbox api on macOS there is already this branch: https://github.com/galad87/HandBrake/tree/videotoolbox
But like sr55 wrote, the encoder supports only average bitrate.

[chriztrax25]

Hey guys, I heard High Sierra provides new APIs for hardware encoding with h265 especially with some of the new intel GPUs. Is this enhancement going to cover that? thanks!

[mikumi]

@chriztrax25 I like your suggestion.

I can confirm that High Sierra is using the discrete graphics card on the MacBook Pro 15" Touchbar 2016+2017 models for encoding. It's pretty fast, and the results look pretty good to my eye. Having support for this in handbrake would be incredibly awesome :)

[wiomoc]

955

[sampumon]

@mikumi which program encodes with GPU?

I notice (MacBookAir 2014, Intel i7 + HD Graphics 5000) that only QuickTime Player does a super-fast GPU encode (uses ~20% CPU).

Not even other Apple programs, like Final Cut Pro, encode as fast. Hence, I wrote a quick'n'dirtyTime batch encoder : )

[mikumi]

@sampumon Sorry for the late reply.

You are right, only QuickTime Player does GPU accelerated encoding, which is very fast and looks pretty good to me.

[panbeep]

I've tried to build the galad87 branch but ended with few errors like:

make[1]: * [/Users/beep/HandBrake/download/x264-snapshot-20160920-2245-stable.tar.bz2] Error 1
ERROR: download failure; ; continuing.

Can somebody provide the compiled file?

[sr55]

It's a network / tooling error on your end. It can't connect to the internet to download files.

Regardless, the patch hasn't been maintained and likely won't be the approach we take to implement it so it's not worth your time. We will be looking at using the ffmpeg API instead most likely as it minimises code we need to implement.

[panbeep]

Tried FFmpeg with Apple videotoolbox acceleration and it's twice that fast compared to CPU rendering only.

[mikumi]

@panbeep Nice, which mac + GPU did you use?

[panbeep]

@mikumi I've tested it on custom build Hackintosh with i7 8700k CPU (Intel UHD 630 GPU + Nvidia 970GTX)

[afanjul]

@panbeep could you post the exact command to hevc encoding with GPU using ffmpef, im having crashes all the times

[sr55]

No-one here has said it's not possible. Adding quicksync support is mostly trivial.
It's more a question of whether we want it or not. Since Apple has hidden the power of Quicksync behind a highly abstract API, that doesn't even allow quality based encoding, it's hard to come up with a good justification to include it.

No concrete decision has been made on this.

As far as Apple/others are concerned, in all likelyhood they are using that same crippled API (In which case, their "pro" tools are not very "pro") or they are using a private API to access it that we can't use. Given the sheer number of people that use HandBrake for post-process encoding, rather than exporting from FCP, Adobe etc due to better quality and file sizes I suspect it's the former.

[sr55]

Please don't put words in my mouth that I didn't say. Read more carefully.

First, it does matter. It's our choice whether we decide if a feature is a good fit for HandBrake or not. The number of people asking for a feature is only one aspect of that decision and doesn't overrule other concerns.

Second, I never said use Ultrafast. What I said was Superfast is close enough and if folks are happy with VT compromises, then using a faster x264 preset is a good measure until we decide either way whether to incorporate this.

Third, HandBrake Core (including sync), Video Decode, Filters, Audio, Subtitles will all still run on the CPU. If the patch is accepted, it's not going to be a full code path which would allow for the ultra low power encoding. Most of HandBrakes features are simply not implemented in hardware.

Fourth, I said I was going to hook up via the ffmpeg interface (also uses VT). I didn't say that code would get committed. It may still be rejected. I want working prototype so I have a clean set of numbers to base a decision on.

[sr55]

Again, putting words in my mouth ...

1. I didn't say QuickSync was trivial. I said it was trivial to implement. (i.e easy to implement)
2. I don't dislike QuickSync. It's arguably the best hardware encoder available in the pro/consumer market. I do use this from time to time on Windows for my non archival use cases and it works very well for this.
3. We literally have thousands of prosumers and professionals using HandBrake to avoid using the substandard H.264 output that these tools provide. Particularly on Mac.
   QuickSync on Windows, runs circles around any implementation of it on Mac. You can achieve much better output with quicksync in video editors on windows (where they offer control)

[bradleysepos]

@dnywlsh Another accusatory retort like your last two and I'm banning you. Please read https://github.com/HandBrake/HandBrake/blob/master/CODE_OF_CONDUCT.md

[panbeep]

@afanjul I'm using ffWorks for this

[afanjul]

@panbeep yeah im using ffworks too, but whenever I tried with HEVC and hardware accelerator it crash or dont write nothing 😞 could you post a screenshot of your config ? plis?

[panbeep]

@afanjul I don't remember now. Probably some default ones. I'm away now so I won't be able to check in two weeks.

[Nomis101]

Could somebody explain me how macOS Videotoolbox encoding actually works? After reading at Apple Developer page I had the meaning it works by using the GPU for acceleration. But after trying out the patch from #1522 I have the feeling the speed improvement is mainly due to disabling of any video optimizations. With my standard settings one BlueRay movie in HEVC takes around 22 (faster setting) or 40 (slow setting) hours. With the patch and Videotoolbox it goes down to 1 hour. Which is nice. But I noticed two drawbacks:
First, the video optimization settings don't work. There is no setting fast, medium, slow and so on. And oll the Command Line Options don't work either. For example the HB log looks like this:

[13:29:42] sync: expecting 190032 video frames
[13:29:42] encavcodecInit: H.265 (VideoToolbox)
[13:29:42] encavcodec: encoding at constant quantizer 1062
[13:29:42] encavcodec: encoding with stored aspect 1/1
[13:29:43] encavcodecInit: Unknown avcodec option rd
[13:29:43] encavcodecInit: Unknown avcodec option strong-intra-smoothing
[13:29:43] encavcodecInit: Unknown avcodec option b-adapt
[13:29:43] encavcodecInit: Unknown avcodec option analysis-reuse-level
[13:29:43] encavcodecInit: Unknown avcodec option limit-tu
[13:29:43] encavcodecInit: Unknown avcodec option lookahead-slices
[h264 @ 0x7f80a18b7a00] sps_id 1 out of range
[13:29:43] sync: first pts audio 0x1 is 0
[13:29:43] sync: first pts audio 0x1 is 0
[13:29:43] sync: first pts audio 0x2 is 0
[13:29:43] sync: first pts audio 0x2 is 0
Isn't it possible to use normal x265 Command Line Options for VideoToolbox encoding or is it just different syntax?

Second thing is, the resulting video looks really horrible. Like in the very early days of video compression. I know, this patch is just a draft and quality will improve, but how does this video toolbox thingy work? Is it just disabling any optimization all together or is there more behind it? Thanks for clarifying.

[sr55]

VideoToolbox is Apples Abstract API for hardware encoder(s). It does not in any way accelerate x264 or x265 encoders and as such, you can't use options for those encoders with it.

QuickSync itself, supports a subset of what x264/5 does. VideoToolbox is yet another subset on that subset. Infact, VT provides virtually no control over the encoder so your ability to tune it is pretty much non existent.

If your on the latest Intel CPU's. 6th gen or later QS on windows produces respectable results. The results will be not so good on macOS though :(

I know, this patch is just a draft and quality will improve

No, quality won't improve. You need to throw more bit-rate at it to achieve better quality. Note VT doesn't support quality encodes (the hardware does though, just limited by apple), only bit-rate based.

[Nomis101]

OK, thanks for clarification. Now my hopes in VideoToolbox are a bit shattered. Because my GPU is good I was hoping QS on the Mac could do the trick. My CPU is an Intel(R) Core(TM) i7-6700HQ, so I think it is 6th gen. But to install Windows on my Mac is not really on option for me. Then I need to bite the bullet and wait 22 h for the encoding. Hope Apple will improve this API.

[sr55]

Apples API is an abstraction over a modest number of encoders for their different platforms. So in all likelyhood it's designed for the lowest common denominator. I'd like to be wrong, but I'm not hopeful in this case.

[tracker1]

I'm still hoping to see NVEnc on macOS at some point. :-(

[sr55]

Doesn't seem likely Apple will support this.

[tracker1]

@sr55 they don't have to, Nvidia would need to though. I'm currently using a hackintosh on my desktop (i7-4790K,. GTX-1080), and would love to be able to do nvenc encodes. But I don't think it's popular enough, which is mildly surprising.

[Nomis101]

No, quality won't improve. You need to throw more bit-rate at it to achieve better quality. Note VT doesn't support quality encodes (the hardware does though, just limited by apple), only bit-rate based.

You are right. I played a bit with VideoToolbox encoding. And per default the bit-rate is very low and so the quality. The only way I could improve quality is to increase the bit rate:
ffmpeg -i video_in.mkv -acodec aac -vcodec hevc_videotoolbox -b:v 6000k -tag:v hvc1 -profile main video_out.mp4
So I could achieve 16x increased video encoding speed and a watchable video quality (however I can spot some motion artifacts). Therefore I think # 1522 with faked bitrate values will give a good compromise between speed and quality.

[galad87]

Added in 5c127e80911b92a8a47b08a51e8dfb52e67fc206