Handbrake: Frame artifacting for lossless sources from OBS

[d4rkne55]

Description of problem or feature request

I'm currently trying to achieve lossless gameplay recording with OBS and I got the recording part working, but when I encode the files with Handbrake, it shows artifacts, like when there were decoding errors/corrupted source. The source seems to be alright, though, at least it looks alright when I play it with VLC.

I tried moving the source to the SSD and encoding from there, with the thought, that the HDD is too slow to feed Handbrake, but that didn't change anything - same result.

I also tried changing the color matrix in OBS to BT.709, because Handbrake uses that, but OBS was using BT.601 by default - also didn't help, though.

Screenshots of the artifacts and the MediaInfo output on the source file under the fifth heading.

Note: Is this maybe related to #1325?

Steps to reproduce the problem

Record gameplay with OBS v21 at lossless settings via NVENC H.264.

HandBrake version (e.g., 1.0.0)

1.0.7

Operating system and version (e.g., Ubuntu 16.04 LTS, macOS 10.3 High Sierra, Windows 10 Creators Update)

Windows 10 (build #1709)

Error message text or screenshot

Artifact examples (JPG): https://imgur.com/a/iTtluym
Artifact examples (PNG): http://privateprojects.w4f.eu/sites/uploads/handbrake-obs-artifacts1.png, http://privateprojects.w4f.eu/sites/uploads/handbrake-obs-artifacts2.png
Source info: https://pastebin.com/CVgearkN

HandBrake Activity Log required (see https://handbrake.fr/docs/en/latest/help/activity-log.html)

https://pastebin.com/JEKbbTuU

[woodstockathbf]

Do you get the same results when using the current version, 1.1.0, or the latest nightly build?

[d4rkne55]

Could try that out, but I wouldn't want to replace 1.0.7 yet and two versions are not supported I heard. Will also try it with ffmpeg CLI when I'm back at home.

On the other side: what about the video files? Can you make use of the encoded file, or the source?

[woodstockathbf]

You can have as many versions as you want, so long as they're in different directories. I keep a half dozen active versions, because some projects need features that were removed from later versions (generation of CLI command lines).

[d4rkne55]

Well, I had that, too, but when I opened a ticket and said that I have two versions, Bradley said that this isn't supported. But, yeah.^^

[sr55]

On Windows, you can run a nightly build alongside a normal build (Support for this was added to the nightly builds last year.

While you can run 1.0.x along side 1.1.x . etc, it's not formally supported and might cause some problems with presets, queues etc if you don't manually manage the config files.

A short sample that reproduces this would be helpful.

[d4rkne55]

Wow.. I just tried the Nightly with the same settings. The encoding is flawless.
That bugs me.. I dislike the GUI changes to be honest, I would rather stay at 1.0.7 after I've seen the Nightly.
Could it be an upstream bug? I don't really think so, though, as I have pretty much the same ffmpeg CLI version that 1.0.7 uses and after a short simple encode it doesn't produce artifacts.. Will test that more thoroughly tomorrow.

I will also upload a sample source tomorrow, not the same I initially gave info about, but one that also results in that artifacts; it's just half of the size, though.^^

[bradleysepos]

HandBrake uses Libav but we are switching to FFmpeg soon.

[d4rkne55]

Well, ok, I was more referring to the x264 version instead of ffmpeg, but thanks for the info. :)

Finally the upload has completed: https://files.fm/u/z7bmbbgv#_

I also tested again via ffmpeg CLI and correct settings, matching the one from Handbrake. It also doesn't produce these artifacts. Version was 148 r2744 b97ae06, Handbrake 1.0.7 used 148 r2708 86b7198.

[d4rkne55]

Oh, the GUI changes are pretty alright btw, I take back what I said yesterday.^^
I didn't have too much time though and then something like a new "Summary" tab, a moved destination input and no preset sidepanel gets on me.^^ And it bugged me, that I couldn't set the resolution to 1440p, but I was on the default 'Fast 1080p30' preset and that seems to be on purpose and was so in 1.0.7, too.

But else, new, unified icons, bold 'Title', 'Angle' and 'Range' headings, destination input at the end actually makes sense from the work flow side, unified 'Add' button style under Audio & Subtitles, better placing of the checkboxes under Subtitles, maybe more. 👍

[cehoyos]

This was fixed in avconv since the release of version 12 and for FFmpeg in 2014 (version 2.3).

[d4rkne55]

Okay, good to know. But I haven't had these problems with other sources before, so what was the reason for this? Didn't it handle lossless sources correctly, was it some combination of settings, some unexpected markers/flags?

I would be interested in the cause for this, maybe also the commit. At least if it doesn't require much effort.

[bradleysepos]

Unfortunately, it does require a bit of effort to track down. Not worth the time to bisect, in my opinion. Nevertheless, glad it's sorted for you. 😸

[d4rkne55]

Okay, alright. I'll take a look into the v12 changelog of libav then and I'm already searching for some git repo.^^

[cehoyos]

This was the original fix:
http://git.videolan.org/?p=ffmpeg.git;a=commitdiff;h=6f7ca1f5
x264 had a bug in lossless encoding for a very long time. Since x264 was the only h264 encoder supporting lossless encoding, FFmpeg contained the same bug. When it was found, both the encoder and the decoder were fixed. Newer files encoded with lossless x264 also need a fixed decoder.
Only one prediction is affected iiuc, this is why you don't see the issue with every file.