Openshot-qt: Incorrect frame in mov file; flickering in export and playback

Created on 5 Mar 2018 · 28Comments · Source: OpenShot/openshot-qt

Operating System: Ubuntu 16.04 x86_64
OpenShot Version: 2.4.1, Latest GIT

Incorrect frame causes flickering in playback and exported result.

How to reproduce:

Drag example movie file(https://www.dropbox.com/s/1rrdgrpguipzd6a/MVI_0448.MOV?dl=0) to the timeline.
Use zoom 1s
Seek to time 11:01
Use arrow keys to move frame by frame
Frame 11:07 is certainly wrong frame. As you see, carousel is moving backward in that frame. The same issue will appear in other files made with same camera, too. Looks like OpenShot is using some data from previous frame in this situation.

Source

croco3008

Most helpful comment

I did some tests using two different computers(ubuntu 16.04 and ubuntu
18.04). Openshot is installed using apt repository. The issue is still
there, but it looks a little bit different. On ubuntu 16.04 the wrong
frame. is the previous frame and it causes a heavy flickering. On ubuntu
18.04 the wrong frame is a stalled frame and the flickering is not so bad.
I will do more test using appimage tomorrow.

2018-07-02 12:23 GMT+03:00 DylanC notifications@github.com:

@croco3008 https://github.com/croco3008 Can you try v2.4.2 and let us
know if you have any success?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/OpenShot/openshot-qt/issues/1336#issuecomment-401730101,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AXs1aglsnFi6mUnVaV0PUqMGStFrlJ4kks5uCeZ8gaJpZM4ScMF4
.

croco3008 on 4 Jul 2018

👍2

All 28 comments

It could just be that the camera shoots using x-fps but OpenShot profile is set to y-fps. Could you please try figuring out the fps of the original footage and use the same profile on OpenShot to edit it?

peanutbutterandcrackers on 6 Mar 2018

Both fps values are equal (25fps). I was using profile HDV 1920x1080p 25fps. Then I tried different profiles(HD 1920x1080i/p 25/24 fps) and the result differed a little bit. It is clear that frame 11:07 is missing at some reason. Difference is that some times the last frame is used to fill missing frame(freeze effect in result) and some times the frame before last is used(Heavy flickering in result).

croco3008 on 6 Mar 2018

Hmm... I see. Well, let me call @DylanC. He knows more than me. And perhaps @ferdnyc too. Together, they might be able to provide some insight into this. :+1:

peanutbutterandcrackers on 6 Mar 2018

Thanx! I Would be very happy if this bug will be fixed! PS: Example movie file can be downloaded by clicking on "..." Icon and choosing "Download".

croco3008 on 6 Mar 2018

Based on your description, I don't _seem_ to be seeing the same thing on my system. (Fedora 27, self-built OpenShot 2.4.1 and python3-libopenshot-0.1.9, atop rpmfusion ffmpeg-libs-3.3.6-1.fc27 and Fedora python3-qt5-5.9.1-2.fc27.)

cast.zip

But, you tell me. That zipped video (OGV converted to MP4, only 1.4MB) is a screencast of the OpenShot interface as I step through the file frame-by-frame from around 00:00:10:02 to 00:00:12:something, by just repeatedly hitting →. It's not super clear and I went kind of quickly, but I don't see any obvious discontinuities; the file seems to preview for me in OpenShot the same as it played in avidemux (where I also tried it). Based on what I can see I feel like I'm not experiencing the same issue you are.

If you agree, try this: Launch the video in ffmpeg's player with ffplay MVI_0448.MOV, and see if you notice the same issue there. (You can hit s to activate frame-by-frame mode, then each successive s will advance one frame. Other controls are listed if you run ffplay --help.)

If you do see the same issue there, then it's your local ffmpeg that's decoding the file improperly (including within OpenShot). If the file plays fine in ffplay, but OpenShot still has issues with it, then we'll have to look into it further.

ferdnyc on 6 Mar 2018

Wait, I think I am seeing it... but at _SIXTEEN_:07, not 11:07. Let me know if this is the sort of thing you saw, and please confirm that it's in a _different_ location in the video than where you saw it. (And for the record, the "skip" I'm seeing does not seem to show up when I play the file in ffplay.)

cast2.zip

ETA: Interestingly, I rendered the project in those casts to HD 720p 25FPS MP4, then loaded the output into avidemux3_qt5 to step through it frame-by-frame. I _don't_ see the discontinuity at ~16 seconds that the OpenShot preview was showing me, but I _do_ see one from 00:00:10.400 to 00:00:10.520 (avidemux timecodes).

ferdnyc on 6 Mar 2018

Hmm. If I'm using profile HD 720p then frame freeze will happen in different position. Using profile HDV 1080p it will happen at 11.07. Using profile 720, first time it will happen at 2:17. At 16:07 is ok for me, but 17:02 is broken again. It could be a memory management bug. By looking your recording, I think it's the same error at 10:400.

croco3008 on 6 Mar 2018

What interests me is that the jitters/discontinuities even corrupt the preview cache — it's not just a display bug. (Too bad, that'd probably be easier to chase down.)

If I load the cached preview frames $HOME/.openshot_qt/preview-cache/${framenum}.ppm (or whatever extension matches the format in Preferences -> Cache) in an image viewer like gThumb, and I flip through the images, I can see the same discontinuity between successive cached frames. So, looks like this is originating in the decoding stage, somewhere, when the frames are being read from the video to build the preview cache. OpenShot is just displaying the cached frames it has to work with, not all of which are properly ordered right from the start.

ferdnyc on 6 Mar 2018

👍1

@DylanC and @N3WWN, I think you two would find the deductions provided here rather interesting. @ferdnyc has done quite an amazing detective work here.

peanutbutterandcrackers on 9 Mar 2018

@croco3008 - I wonder if it is an issue with the .MOV extension. Could you perhaps try and transcode the video using HandBrake and see if the same issue persists even in say, a .MP4 or a .MKV clip? o.O

peanutbutterandcrackers on 9 Mar 2018

The issue is evident on my Arch Linux Rig. It renders correctly frame by frame using mpv, but in OpenShot the issue is there. However when I use ffmpeg and trancode it (h264 mkv ac3) and then seems to play correctly in Openshot.

Camera files are not always great. I have had issues with playback on certain devices.

gary9872 on 11 Mar 2018

@gary9872 - Could you perhaps patch this too? o.O Sorry if I'm asking for much...

peanutbutterandcrackers on 11 Mar 2018

@peanutbutterandcrackers I wish I was that smart.

To add to ferdnyc suggestion, I would suggest try playing it in MELT and advance it frame by frame.

Interestingly, I can't er CAN produce the error, but NOT consistently.

About the 12 second mark, frames 1-3

https://youtu.be/BbcJ0uHE0-Y

gary9872 on 11 Mar 2018

@gary9872 - So do I. But at least you have submitted some profile thingies. That's a great start. Hopefully you'll be able to submit patches soon, too.

I don't know much about this issue, either... I wish I could help, though...

peanutbutterandcrackers on 12 Mar 2018

Openshot doesn't seem to be caching correctly.

While moving frame by frame, I watched the frames being added to the cache. I would guess that it not normal when moving one frame to add over 10 frames to the cache, and then add a pile more moving to the next frame, and again. (Some frames were cached more than once moving only 2 frames)

gary9872 on 13 Mar 2018

👍1

@croco3008 Could you go to preference and set the Cache Limit to zero and see if the problem persists?

gary9872 on 14 Mar 2018

@gary9872 - Do you think incorrect caching is actually a bug? I mean, is it reproducible for other videos too? I haven't really noticed this issue.

@DylanC - If it is indeed incorrect caching, perhaps it needs some major attention. Does this problem arise for you too, Cap'ain? I haven't really noticed this...

peanutbutterandcrackers on 14 Mar 2018

I did few tests with Cache Limit set to zero. The issues still exists and looks similar like cache set to non zero value. By my mind the bug lays somewhere in libopenshot.

croco3008 on 14 Mar 2018

@croco3008 Did you close openshot, then start a new project and re-import your video? Just trying to be thorough.

For me I can replicate it only with the cache on.

When I'm going frame I frame and everything is well (see below)

BUT when all is not well when this happens (https://pastebin.com/ZX0RXTXZ) and the frames get out of order.

In the PASTE you can see it adding frames to the cache:

437,435,440,441,443,434,436,442,444,439,433 seemingly in that order

This ALL happens in advancing a frame. I dunno perhaps this is normal, but this is when the bug appears ( at least for me)

Good output from frame by frame.

logger_libopenshot:INFO Timeline::GetFrame (Cached frame found) (requested_frame=436.0000)
logger_libopenshot:INFO PlayerPrivate::run (determine sleep) (video_frame_diff=0.0000, video_position=436.0000, audio_position=436.0000, speed=0.0000, render_time=1.0000, sleep_time=39.0000)
logger_libopenshot:INFO VideoPlaybackThread::run (before render) (frame->number=436.0000, need_render=1.0000)
logger:INFO 16777234
main_window:INFO keyPressEvent: Left
logger_libopenshot:INFO Timeline::GetFrame (Cached frame found) (requested_frame=435.0000)
logger_libopenshot:INFO PlayerPrivate::run (determine sleep) (video_frame_diff=0.0000, video_position=435.0000, audio_position=435.0000, speed=0.0000, render_time=0.0000, sleep_time=40.0000)
logger_libopenshot:INFO VideoPlaybackThread::run (before render) (frame->number=435.0000, need_render=1.0000)
logger:INFO 16777234

gary9872 on 15 Mar 2018

Yes, I did. I did some more tests and for me the bug is still there. I can see both - the freezeframes and frames in wrong order.
My settings:
https://www.dropbox.com/s/t68boef0ddni1lh/settings.png?dl=0

croco3008 on 16 Mar 2018

@croco3008 The only other thing I could suggest (if you haven't already done this version) is try the one of the AppImages https://www.openshot.org/download/ Click on the daily builds button and try one of those, as they are I believe self contained and don't rely on external video/audio libraries.

I know you have mentioned you have tried GIT, but I don't know if the is a PPA or and AppImage. You might do better with an AppImage. Cheers,

gary9872 on 16 Mar 2018

I hoped that the following commit could fix this issue, but it couldn't.
https://github.com/OpenShot/libopenshot/commit/54d6cf599576030416bf9d734eb6dc9b3058640c

croco3008 on 20 Apr 2018

@gary9872 :

For me I can replicate it only with the cache on.

When I'm going frame I frame and everything is well (see below)

BUT when all is not well when this happens (https://pastebin.com/ZX0RXTXZ) and the frames get out of order.

In the PASTE you can see it adding frames to the cache:

437,435,440,441,443,434,436,442,444,439,433 seemingly in that order

This ALL happens in advancing a frame. I dunno perhaps this is normal, but this is when the bug appears ( at least for me)

I don't _think_ that's an issue, and I suspect it is indeed normal.

Your log's too truncated to really get a clear picture, but I suspect what you're seeing is simply OpenShot performing operations in parallel. By definition, parallel tasks are not executed in any specific order (they're all executed at once), so they're not guaranteed to complete in any particular order. (In fact, in practice they're guaranteed _not_ to always complete in a given order order, or in the same order every time. When they do, it's merely a coincidence.)

I have a four-core processor, so grepping for "thread" in my $HOME/.openshot_qt/libopenshot.log results in tons of blocks like this:

Timeline::GetFrame (processing frame) (frame_number=173.0000, omp_get_thread_num()=0.0000)
Timeline::GetFrame (processing frame) (frame_number=175.0000, omp_get_thread_num()=2.0000)
Timeline::GetFrame (processing frame) (frame_number=174.0000, omp_get_thread_num()=1.0000)
Timeline::GetFrame (processing frame) (frame_number=176.0000, omp_get_thread_num()=3.0000)
Timeline::GetFrame (end parallel region) (requested_frame=173.0000, omp_get_thread_num()=0.0000)
Timeline::GetFrame (processing frame) (frame_number=1.0000, omp_get_thread_num()=0.0000)
Timeline::GetFrame (processing frame) (frame_number=2.0000, omp_get_thread_num()=1.0000)
Timeline::GetFrame (processing frame) (frame_number=3.0000, omp_get_thread_num()=2.0000)
Timeline::GetFrame (processing frame) (frame_number=4.0000, omp_get_thread_num()=3.0000)
Timeline::GetFrame (end parallel region) (requested_frame=1.0000, omp_get_thread_num()=0.0000)
Timeline::GetFrame (processing frame) (frame_number=1.0000, omp_get_thread_num()=0.0000)
Timeline::GetFrame (processing frame) (frame_number=4.0000, omp_get_thread_num()=3.0000)
Timeline::GetFrame (processing frame) (frame_number=3.0000, omp_get_thread_num()=2.0000)
Timeline::GetFrame (processing frame) (frame_number=2.0000, omp_get_thread_num()=1.0000)
Timeline::GetFrame (end parallel region) (requested_frame=1.0000, omp_get_thread_num()=0.0000)
VideoPlaybackThread::run (before render) (frame->number=1.0000, need_render=1.0000)
Timeline::GetFrame (processing frame) (frame_number=1.0000, omp_get_thread_num()=0.0000)
Timeline::GetFrame (processing frame) (frame_number=3.0000, omp_get_thread_num()=2.0000)
Timeline::GetFrame (processing frame) (frame_number=4.0000, omp_get_thread_num()=3.0000)
Timeline::GetFrame (processing frame) (frame_number=2.0000, omp_get_thread_num()=1.0000)
Timeline::GetFrame (end parallel region) (requested_frame=1.0000, omp_get_thread_num()=0.0000)
VideoPlaybackThread::run (before render) (frame->number=1.0000, need_render=1.0000)
Timeline::GetFrame (processing frame) (frame_number=3.0000, omp_get_thread_num()=2.0000)
Timeline::GetFrame (processing frame) (frame_number=1.0000, omp_get_thread_num()=0.0000)
Timeline::GetFrame (processing frame) (frame_number=4.0000, omp_get_thread_num()=3.0000)
Timeline::GetFrame (processing frame) (frame_number=2.0000, omp_get_thread_num()=1.0000)
Timeline::GetFrame (end parallel region) (requested_frame=1.0000, omp_get_thread_num()=0.0000)
VideoPlaybackThread::run (before render) (frame->number=1.0000, need_render=1.0000)

Different ordering every time, and only one that happened to execute in linear frame order.

It sounds like you may have an 8- or even 16-core system, which is why you'll see "scrambled" runs of frames being processed in chunks of N at a time (where N = the number of available CPU cores).

As long as OpenShot is correctly keeping track of which frame is which, the order they're processed in shouldn't matter, and it will probably happen out of order more often than not. It's _possible_ that there is a parallelization issue behind this problem, but if there is I'd expect to see it a lot more than we have, and a lot less repeatably. (In my testing, the stutters were _always_ in the same place, every time. Which isn't likely to be caused by a parallelism race condition. By their very nature, those tend _not_ to be easily predictable/repeatable.)

ferdnyc on 21 Apr 2018

There are multiple threads obtaining frames simultaneously (hence the end parallel region text in the output). GetFrame() frames coming back in seemingly random orders is to be expected.

FFmpegReader will attempt to read as many frames as you have cores in your system.

The above is just me agreeing with @ferdnyc 😉

But, I wonder if the issue has to do with B-frames.... even if using ➡️ to advance frame by frame, it is possible the decoder is going forward to the next keyframe and then reversing to the requested frame.

In the source, it is even noted:

Seek to a specific frame. This is not always frame accurate, it's more of an estimation on many codecs.

This means it's possible a specific frame returned by the decoder to not be the actual frame that you want.

I'd be interested to see if transcoding helps or hurts the issue...

Has anyone checked to see if there are duplicate frame warnings returned by the ffmpeg library? As far as I can tell, the ffmpeg library output is usually colorized and sent to the terminal used to launch OpenShot and _not_ stored in libopenshot.log or openshot-qt.log. (I use the AppImage, BTW, and get a lot of output to the terminal)

N3WWN on 10 May 2018

This means it's possible a specific frame returned by the decoder to not be the actual frame that you want.

Which is still not necessarily a problem, though, as long as the decoder _knows_ that it's not the frame you wanted. (Meaning, if you seek to frame 27 and the decoder spits out frame 28, that's only a problem if it tells you, or you assume, that it's frame 27. If you _know_ it's frame 28, and can adjust accordingly, then it's possible to make things work despite a lack of frame-accuracy. Avidemux does an excellent job with this, it explicitly doesn't pretend that it can always be frame-accurate and just comes as close as it can.)

ferdnyc on 11 May 2018

Has anyone checked to see if there are duplicate frame warnings returned by the ffmpeg library? As far as I can tell, the ffmpeg library output is usually colorized and sent to the terminal used to launch OpenShot and not stored in libopenshot.log or openshot-qt.log. (I use the AppImage, BTW, and get a lot of output to the terminal)

I didn't see any, in my testing. But I wouldn't expect to, because playing the clip in ffplay didn't result in any duplicate frames or stutter. The issue doesn't appear to be the source media, or the decoder.

ferdnyc on 11 May 2018

@croco3008 Can you try v2.4.2 and let us know if you have any success?

DylanC on 2 Jul 2018

2018-07-02 12:23 GMT+03:00 DylanC notifications@github.com:

@croco3008 https://github.com/croco3008 Can you try v2.4.2 and let us
know if you have any success?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/OpenShot/openshot-qt/issues/1336#issuecomment-401730101,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AXs1aglsnFi6mUnVaV0PUqMGStFrlJ4kks5uCeZ8gaJpZM4ScMF4
.