Php-ffmpeg: extractMultipleFrames creates too many frame images than expected

[Shaked]

| Q | A
| -------------- | ---
| Bug? | Yes
| New Feature? | no
| Version Used | ^0.11.1
| FFmpeg Version | ffmpeg version 3.3.3 Copyright (c) 2000-2017 the FFmpeg developers built with Apple LLVM version 8.1.0 (clang-802.0.42)
| OS | OSX 10.33.3

Actual Behavior

How does PHP-FFMpeg behave at the moment?

When I use extractMltipleFrames(ExtractMultipleFramesFilter::FRAMERATE_EVERY_60SEC, $dir) using 1 minutes video, I get 1682 images (frameXXXX.jpg) AND for some reason I also get output_XX files. (mentioned in https://github.com/PHP-FFMpeg/PHP-FFMpeg/issues/493)

NOTE
For some reason I have to do $video->save($format, $path); at the end (see full code below). This ends up with frame*.jpg & output_*.png while the .png files are irrelevant in this case. Is it by design?

Expected Behavior

What is the behavior you expect?

For 1 minute video I'd expect to have 60 images (1 per second).

Steps to Reproduce

What are the steps to reproduce this bug? Please add code examples,
screenshots or links to GitHub repositories that reproduce the problem.

$ composer require php-ffmpeg/php-ffmpeg
$ curl -O https://gist.githubusercontent.com/Shaked/98bd85babc56f3b68605c2165dfc8797/raw/f496d59e2f6f2e1471e8f12aaebde9d4e02f1f53/php-ffmpeg.php
$ php php-ffmpeg.php
$ ls -la /tmp/test_dir

Possible Solutions

If you have already ideas how to solve the issue, add them here.
Otherwise remove this section.

[jens1o]

Thank you for the report! Will look into this in the next days, I'm in a hurry atm. Feel free to ping me when I don't reply until Thursday. :)

[Shaked]

BTW I have just ran:

$ ffmpeg -i 'https://instagram.fmex7-1.fna.fbcdn.net/vp/1417659ca05109537276143f6fa427c2/5AA336A7/t50.2886-16/26162850_164145864357402_482322447928918016_n.mp4' -q:v 1 -vf fps=1 ./frame%04d.png

and I get 57 images which makes sense cause the video is 57 seconds 🙂

EDIT:

While debugging I found that ExtractMultipleFrameFilter's line 127 disappears from the executed commands. I.E

This: $commands[] = $this->destinationFolder . 'frame-%' . $nbDigitsInFileNames . 'd.png';
Is not available in: Video:73:

$passes = $this->buildCommand($format, $outputPathfile);
var_dump($passes);

[42]=>
string(3) "-vf"
[43]=>
string(17) "[in]fps=1/60[out]"
[44]=>
string(5) "-pass"
[45]=>

EDIT 2:

OK the line is there, just not next to string(17) "[in]fps=1/60[out]"

[jens1o]

So, do you have a point where I can step into?

[Shaked]

I have few potential ideas but not sure what's the source currently:

1. Some of the passes is messing up -vf fps=1/X path
2. The save() method appends another path, maybe that's the problem?
3. Going far but maybe the fps=1/60 which means "one frame every minute" becomes 0,016? that would create a huge amount of frames cause it means "one frame every 0,016seconds"

[jens1o]

Alright, that is very helpful. Thank you! I'm going to investigate, now.

[jens1o]

I can confirm this, but tbh, I don't now why ffmpeg declines with Could not open file : output/frame-01.jpg.

[Shaked]

I can open the frame%d.jpg/png files but can't open whatever comes from $video->save(format, 'output%d.jpg').

I'd start with removing passes and see if it changes anything. Also, is it possible to do somehow do $video->save without saving another file?

[jens1o]

I'd start with removing passes and see if it changes anything. Also, is it possible to do somehow do $video->save without saving another file?

No, because we somehow need to track its progress in a general way. Literally what I'm doing:

1. Set up my env
2. Download your test video directly to avoid long running scripts(just wastes time).
3. Modify the last section to

$finalCommand = $video
    ->getFinalCommand($format, $path);

var_dump($finalCommand);

1. Copy out the first command in that array.
2. Prepend ffmpeg to it, remove the pass things and let it run.
3. Get confused about the result 🐛

So atm, I'm trying to find the parts of the final command which are breaking these things.

--

I can open the frame%d.jpg/png files but can't open whatever comes from $video->save(format, 'output%d.jpg').

I'm not trying to open then, that's the error of ffmpeg itself(not of this wrapper!)

[jens1o]

Ah, okay.

Got it!

We're setting the output thing twice, so instead of

ffmpeg -y -i test-video-494.mp4 output/frame-%02d.jpg -vcodec libx264 -acodec libfaac -b:v 1000k -refs 6 -coder 1 -sc_threshold 40 -flags +loop -me_range 16 -subq 7 -i_qfactor 0.71 -qcomp 0.6 -qdiff 4 -trellis 1 -b:a 128k -vf [in]fps=1/60[out] output/output_%d.png

we should use

ffmpeg -y -i test-video-494.mp4 -vcodec libx264 -acodec libfaac -b:v 1000k -refs 6 -coder 1 -sc_threshold 40 -flags +loop -me_range 16 -subq 7 -i_qfactor 0.71 -qcomp 0.6 -qdiff 4 -trellis 1 -b:a 128k -vf [in]fps=1/60[out] output/output_%d.png

Note that we're only giving it one output stream and perhaps ffmpeg thinks we give two inputs to it.

--

Perhaps you want to fix it? It's an easy bug after the first look. :)

[Shaked]

Perhaps you want to fix it? It's an easy bug after the first look. :)

No problem. I am just gonna remove the frame-%d output basically. Correct?

EDIT:

I am playing with my example while I removed this line:

$commands[] = $this->destinationFolder . 'frame-%' . $nbDigitsInFileNames . 'd.png';

When executing print_r($video->getFinalCommand($format, $path), true) I get 2 commands for some reason which end up with:

Array
(
    [0] => -y -i https://instagram.fmex7-1.fna.fbcdn.net/vp/1417659ca05109537276143f6fa427c2/5AA336A7/t50.2886-16/26162850_164145864357402_482322447928918016_n.mp4 -async 1 -metadata:s:v:0 start_time=0 -vcodec libx264 -acodec libfaac -b:v 1000k -refs 6 -coder 1 -sc_threshold 40 -flags +loop -me_range 16 -subq 7 -i_qfactor 0.71 -qcomp 0.6 -qdiff 4 -trellis 1 -b:a 128k -vf [in]fps=1/60[out] -pass 1 -passlogfile /var/folders/mx/g385rhp53w76tqh1s66_p28h0000gn/T/ffmpeg-passes5aa1c8023e0cbab0rb/pass-5aa1c8023e19c /private/var/www/whattoeatabroad.com/config/../web/videos/73c021fbca8a0164c0e38c5a1e8a78914db04352/output_%d.png
    [1] => -y -i https://instagram.fmex7-1.fna.fbcdn.net/vp/1417659ca05109537276143f6fa427c2/5AA336A7/t50.2886-16/26162850_164145864357402_482322447928918016_n.mp4 -async 1 -metadata:s:v:0 start_time=0 -vcodec libx264 -acodec libfaac -b:v 1000k -refs 6 -coder 1 -sc_threshold 40 -flags +loop -me_range 16 -subq 7 -i_qfactor 0.71 -qcomp 0.6 -qdiff 4 -trellis 1 -b:a 128k -vf [in]fps=1/60[out] -pass 2 -passlogfile /var/folders/mx/g385rhp53w76tqh1s66_p28h0000gn/T/ffmpeg-passes5aa1c8023e0cbab0rb/pass-5aa1c8023e19c /private/var/www/whattoeatabroad.com/config/../web/videos/73c021fbca8a0164c0e38c5a1e8a78914db04352/output_%d.png
)

When trying to open the output_%d.png files, I get an error - no image.

NOTE also the onProgress method sometimes never gets to 100% but rather to 99% or 98%.

EDIT 2

FOUND IT. When using -vcodec libx264 -acodec libfaac the generated images are corrupted. I guess because it tries to use video codecs for the images?

It is impossible to disable video & audio codecs, so for my temp solution is to have this:

<?php

/*
 * This file is part of PHP-FFmpeg.
 *
 * (c) Alchemy <[email protected]>
 *
 * For the full copyright and license information, please view the LICENSE
 * file that was distributed with this source code.
 */

namespace FFMpeg\Format\Video;

/**
 * The X264 video format
 */
class Simple extends DefaultVideo {
    /** @var boolean */
    private $bframesSupport = true;

    /** @var integer */
    private $passes = 1;

    /**
     * {@inheritDoc}
     */
    public function supportBFrames() {
        return $this->bframesSupport;
    }

    /**
     * @param $support
     *
     * @return X264
     */
    public function setBFramesSupport($support) {
        $this->bframesSupport = $support;

        return $this;
    }

    /**
     * {@inheritDoc}
     */
    public function getAvailableAudioCodecs() {
        return [];
    }

    /**
     * {@inheritDoc}
     */
    public function getAvailableVideoCodecs() {
        return [];
    }

    /**
     * @param $passes
     *
     * @return X264
     */
    public function setPasses($passes) {
        $this->passes = $passes;
        return $this;
    }

    /**
     * {@inheritDoc}
     */
    public function getPasses() {
        return $this->passes;
    }

    /**
     * @return int
     */
    public function getModulus() {
        return 2;
    }
}

$path = sprintf('%s/output_%s.png', $dir, '%d');
$format = new \FFMpeg\Format\Video\Simple();
$format->on('progress', function ($video, $format, $percentage) {
error_log('percentage: '. $percentage);
});
 $video
                ->save($format, $path);

Which works as expected! Final command is:

ffmpeg -y -i https://instagram.fmex7-1.fna.fbcdn.net/vp/1417659ca05109537276143f6fa427c2/5AA336A7/t50.2886-16/26162850_164145864357402_482322447928918016_n.mp4 -b:v 1000k -refs 6 -coder 1 -sc_threshold 40 -flags +loop -me_range 16 -subq 7 -i_qfactor 0.71 -qcomp 0.6 -qdiff 4 -trellis 1 -b:a 128k -vf [in]fps=1/1[out] /private/var/www/whattoeatabroad.com/config/../web/videos/73c021fbca8a0164c0e38c5a1e8a78914db04352/output_%d.png

[jens1o]

Awesome, could you please file a PR(with tests) as soon as the current one has been approved? Please do not do it yet, because your first PR is based on the master branch.

[Shaked]

Awesome, could you please file a PR(with tests) as soon as the current one has been approved? Please do not do it yet, because your first PR is based on the master branch.

Yes I will start making one and file a PR this week. Thank you

[vincenzodibiaggio]

From the version 0.11.1 I've resolved this issue with this patch

diff --git a/src/FFMpeg/Filters/Video/ExtractMultipleFramesFilter.php b/src/FFMpeg/Filters/Video/ExtractMultipleFramesFilter.php
index d0ed54f..0cbee70 100644
--- a/src/FFMpeg/Filters/Video/ExtractMultipleFramesFilter.php
+++ b/src/FFMpeg/Filters/Video/ExtractMultipleFramesFilter.php
@@ -89,35 +89,10 @@ class ExtractMultipleFramesFilter implements VideoFilterInterface
                 }
             }

-            // Get the number of frames per second we have to extract.
-            if(preg_match('/(\d+)(?:\s*)([\+\-\*\/])(?:\s*)(\d+)/', $this->frameRate, $matches) !== FALSE){
-                $operator = $matches[2];
-
-                switch($operator){
-                    case '/':
-                        $nbFramesPerSecond = $matches[1] / $matches[3];
-                        break;
-
-                    default:
-                        throw new InvalidArgumentException('The frame rate is not a proper division: ' . $this->frameRate);
-                        break;
-                }
-            }
-
-            // Set the number of digits to use in the exported filenames
-            $nbImages = ceil( $duration * $nbFramesPerSecond );
-
-            if($nbImages < 100)
-                $nbDigitsInFileNames = "02";
-            elseif($nbImages < 1000)
-                $nbDigitsInFileNames = "03";
-            else
-                $nbDigitsInFileNames = "06";
-
             // Set the parameters
-            $commands[] = '-vf';
-            $commands[] = 'fps=' . $this->frameRate;
-            $commands[] = $this->destinationFolder . 'frame-%'.$nbDigitsInFileNames.'d.jpg';
+            $commands[] = '-r';
+            $commands[] = $this->frameRate;
+            $commands[] = $this->destinationFolder . 'frame-%d.jpg';
         }
         catch (RuntimeException $e) {
             throw new RuntimeException('An error occured while extracting the frames: ' . $e->getMessage() . '. The code: ' . $e->getCode());

[jens1o]

Would you mind filing a pull request @vincenzodibiaggio ?