Handbrake: CRF mode of VP9 not working

[CKDing]

It gave me very big file size and very long encoding time even if I set to lowest 63.

[woodstockathbf]

Without a log from that encode, there isn't much that anyone can do to help you.

[CKDing]

Thank you for reply. Here is the log: https://pastebin.com/GBiLNBzQ

[frederickding]

I can verify that on a recent (self-built) nightly, a test 480p encode with VP9 at CRF 63 resulted in a video output stream at 8182.57 kbps. This is obviously unexpected behavior (even though nobody should be using CRF 63 anyway).

Log: https://gist.github.com/frederickding/3c1d7ce7a690ba87471cc5d5131a21c6

I think the bug is at https://github.com/HandBrake/HandBrake/blob/065a485ccbaf9fc051e0d6f607ae590f5aaa40d4/libhb/encavcodec.c#L194
This line attempts to set a high bitrate by some arithmetic. The test encode I did suggests that this is the bitrate being used (720 × 480 × 24000/1001 ≈ 8092).

But setting a nonzero bitrate means that ffmpeg will use Constrained Quality instead of Constant Quality. That documentation says "for videos that are 'hard' to encode the quality will be bounded by the maximum bitrate and will behave like the Variable Bitrate mode." This explains why some VP9 encodes end up filling up the defined bitrate from encavcodec.c instead of actually letting the encoder choose the bitrate by CRF.

[sr55]

My guess is "Hard to encode" includes anything that's short (Possibly other scenarios as well) as anything greater than 1 minute works fine for me:

[jstebbins]

Setting the bit_rate is supposed to set an upper bound to the bitrate used when encoding the file. If we don't set this, libvpx uses it's own completely inappropriate setting for an upper bound. It's a bit like VBV settings for x264 and x265. A CRF of 63 should result in a very low bitrate that is nowhere near the upper bound we are computing so this bitrate should not be a factor when using this CRF. I think @sr55 is likely correct, the encoder just does a terrible job of rate control with short files.

[CKDing]

Thank you! You guys are very helpful.