Php-ffmpeg: ExtractMultipleFrames filter generating excessively amount of images

[larzuk91]

| Q | A
| -------------- | ---
| Bug? | Maybe
| New Feature? | no
| Version Used | 0.12.0
| FFmpeg Version | 3.4.4-1~16.04.york0
| OS | Ubuntu 16.04.5

Hi, I just upgraded to version 0.12.0 and the ExtractMultipleFrames filter started to misbehave. I'm processing a 5 seconds long video with a _frame rate_ of 1/1, so it should generate 5 images but instead it's generating 136!

Before upgrading I was using the version 0.9.3 with FFmpeg 2.8 (Everything work as a charm)

This is the code that used to work:

    public function __construct($ffmpegBinary, $ffprobeBinary, $timeout, $threads)
    {
        $this->FFMpeg = FFMpeg::create([
            'ffmpeg.binaries' => $ffmpegBinary,
            'ffprobe.binaries' => $ffprobeBinary,
            'timeout' => $timeout, // The timeout for the underlying process
            'ffmpeg.threads' => $threads,   // The number of threads that FFMpeg should use
        ]);
    }

    public function generateVideoThumbnails($videoUrl, $outputDir, $width, $frameRate)
    {
        $video = $this->FFMpeg->open($videoUrl);
        $video->filters()
            ->extractMultipleFrames($frameRate, $outputDir)
            ->synchronize();

        $video->save(new X264('aac'), $outputDir."/video.mp4");
    }

and this is the final command being run:
/usr/bin/ffmpeg '-y' '-i' '/project/test_files/chapter_video.mp4' '-async' '1' '-metadata:s:v:0' 'start_time=0' '/tmp/ffmpeg/frame-%02d.jpg' '-threads' '4' '-vcodec' 'libx264' '-acodec' 'aac' '-b:v' '1000k' '-refs' '6' '-coder' '1' '-sc_threshold' '40' '-flags' '+loop' '-me_range' '16' '-subq' '7' '-i_qfactor' '0.71' '-qcomp' '0.6' '-qdiff' '4' '-trellis' '1' '-b:a' '128k' '-vf' '[in]fps=1/1[out]' '-pass' '1' '-passlogfile' '/tmp/ffmpeg/pass-5b8729cba1336' '/tmp/ffmpeg/video.mp4'

I don't know what else I should do, I already tried with other codecs like libmp3lame. If I change the frameRate nothing happens, so I believe there might be a bug on the code or an incompatibility between FFmpeg and project's version

Thanks in advance.

[larzuk91]

Oh, and the video generated has its length fine (~5 seconds)

[jens1o]

I'll look into this tomorrow, please be patient.

[larzuk91]

Sure thing @jens1o
If there is something you think I should try in the meanwhile, just let me know

[jens1o]

Yeah, in the meantime you can rollback to 0.9 ;)

[jens1o]

The codebase for this filter really hasn't changed much since the release of 0.9. We added the possibility to define the file extension. Could you try using the latest ffmpeg?

[larzuk91]

Sure, I'll update to the latest version and give it another try

Yeah, in the meantime you can rollback to 0.9 ;)

My real problem is that I'm using Symfony4 and there is an incompatibility version with alchemy/binary-driver dependency, it requires version >= 2.0 and since it was added in the latest commit (Actually, I proposed the change), I need to require master's branch for my project

[larzuk91]

Dang, upgrading to FFMpeg 4.0.2 didn't help. I did some more digging to the class ExtractMultipleFramesFilter and its values look correct ($duration, $nbImages & $nbFramesPerSecond) and also I noticed an interesting thing, I tried with another video which has 1 second more length (5 seconds length total), and from generating 135 images it raised to 160

Oh, and FFMpeg 4.0.2 with PHP-FFMpeg 0.9.3 works correctly

Another note: it only works with version 0.9.3 it fails since 0.9.4 what a coincidence...

[jens1o]

it fails since 0.9.4 what a coincidence...

I feel like I'm stupid, I'm sorry, but thanks for the regression test. :)

0.9.3 has been released on Apr 12, 2017. 0.9.4 has been released on Jun 28, 2017 (without any release note).

But there weren't any updates to this file between these two releases. https://github.com/PHP-FFMpeg/PHP-FFMpeg/commits/0.9.4/src/FFMpeg/Filters/Video/ExtractMultipleFramesFilter.php (note the first(latest) commit in that tag was already released with 0.9.3). Which means either I'm stupid or the problem may lay anywhere in the codebase.

I can't set up any environment at the moment, but could you try what 0.9.3 is applying(after calculation) and what 0.9.4 does? If they are the same, we need to debug the hole codebase.

[jens1o]

(Prepend a var_dump($commands); statement(or work with a real debugger) or anything else here: https://github.com/PHP-FFMpeg/PHP-FFMpeg/blob/cedcd37bd22479bbe7874fd0e063a2047e122d0a/src/FFMpeg/Filters/Video/ExtractMultipleFramesFilter.php#L126 )

[larzuk91]

Sure, I'll try it out tomorrow morning - Thanks for your support :)

[larzuk91]

Sorry I expected to have some free time this weekend but well... here it is the output from both versions (they are equals =/)

==0.9.3==
ExtractMultipleFramesFilter.php on line 126:
array:3 [
  0 => "-vf"
  1 => "fps=1/1"
  2 => "/tmp/thumbnails5b8d57b5c39af5.64543542/frame-%02d.jpg"
]
Generates 5 images

==0.9.4==
ExtractMultipleFramesFilter.php on line 126:
array:3 [
  0 => "-vf"
  1 => "fps=1/1"
  2 => "/tmp/thumbnails5b8d58e9ce7959.86058999/frame-%02d.jpg"
]
Generates 135

[jens1o]

me

[jens1o]

I'll look into it.

[larzuk91]

Thanks man

[jens1o]

Okay, I can (finally) confirm this. There are two commits that may cause this, that have been implemented in the 0.9.4 release:

1. https://github.com/PHP-FFMpeg/PHP-FFMpeg/commit/4cfcabd7b50266f11c00e4c2c39292b15b312c13
2. https://github.com/PHP-FFMpeg/PHP-FFMpeg/commit/cb7d850338844d74f425c44c0b883d3b24681255

I try manually de-cherry-picking them, and hopefully this will resolve this.

[jens1o]

looks like nobody is using this.. :/

It is the commit of https://github.com/PHP-FFMpeg/PHP-FFMpeg/commit/4cfcabd7b50266f11c00e4c2c39292b15b312c13 in the Video.php part.

(not my commit c:)...

This commit does have tests though. I honestly do not understand what it tries to do, and these tests do not fail, they are passing.

[jens1o]

(the strict use of semver and a changelog would have made this a lot easier)

[larzuk91]

Oh, interesting, maybe an integration test would have caught this

looks like nobody is using this.. :/

That's what I thought (How could I be the only one having this issue?), thanks for your help man, I hope the fix would be easy enough :)

[jens1o]

I think about rewriting everything from scratch for 1.x, because the codebase is so... old-fashion. But I do not know if it is worth it, close to nobody uses it. (No, I will not abandon this project.)

I do not understand the need behind the change(/commit), yet. The most trivial method would be to back the change out.

[larzuk91]

Oh man, I completely understand you, doing a rewrite it's always very tempting but it can be a trap because of all the hidden work it may have

[larzuk91]

Hello @jens1o, sorry for disturbing you but do you think this bug will be fixed any time soon?

[jens1o]

I'm still not able to tell what the code is doing there. :| I'll need to check whether it is possible to back the change out.

[larzuk91]

Dang, let me know if you find something

Thanks for your support

[jens1o]

Moving this to the next release, hopefully I'm able to handle this soon, but I'm not available next week. :/

[larzuk91]

Ah ok, I understand, thanks for your time man