Moviepy: ffmpeg 4.0 NVIDIA NVDEC-accelerated Support ?

[zylhub]

ffmpeg 4.0 has released the NVIDIA NVDEC-accelerated H.264, HEVC, MJPEG, MPEG-1/2/4, VC1, VP8/9 hwaccel decoding added.

see more ffmpeg 4.0 NVENC

so, Is it possible to increase support for gpu acceleration?

my ffmpeg

ffmpeg version N-87822-g734ed38 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.4) 20160609
  configuration: --enable-nonfree --disable-shared --enable-nvenc --enable-cuda --enable-cuvid --enable-libnpp --extra-cflags=-I/usr/local/cuda/include --extra-cflags=-I/usr/local/include --extra-ldflags=-L/usr/local/cuda/lib64
  libavutil      55. 79.100 / 55. 79.100
  libavcodec     57.108.100 / 57.108.100
  libavformat    57. 84.100 / 57. 84.100
  libavdevice    57. 11.100 / 57. 11.100
  libavfilter     6.108.100 /  6.108.100
  libswscale      4.  9.100 /  4.  9.100
  libswresample   2. 10.100 /  2. 10.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

Use -h to get full help or, even better, run 'man ffmpeg'

Here is my idea:

1. complied ffmpeg with nvidia video sdk : https://developer.nvidia.com/ffmpeg or use docker image https://hub.docker.com/r/nightseas/ffmpeg/
2. modified the ffmpeg_reader.py in moviepy

moviepy/video/io/ffmpeg_reader.py line89-95

    cmd = ([get_setting("FFMPEG_BINARY")] + i_arg +
           ['-loglevel', 'error',
            '-f', 'image2pipe',
            '-vf', 'scale=%d:%d' % tuple(self.size),
            '-sws_flags', self.resize_algo,
            "-pix_fmt", self.pix_fmt,
            '-vcodec', 'rawvideo', '-'])

I guess the cmd is made up of multiple parameters, so I modified as the following

    cmd = ([get_setting("FFMPEG_BINARY")] + i_arg +
           ['-loglevel', 'error',
            '-f', 'image2pipe',
            '-vf', 'scale=%d:%d' % tuple(self.size),
            '-sws_flags', self.resize_algo,
            "-pix_fmt", self.pix_fmt,
            '-vcodec', 'h264_nvenc',
            '-preset', 'default'])
            #'-vcodec', 'rawvideo', '-'])

BUT, I got ：

 MoviePy error: failed to read the first frame of "
                           "video file test.mp4. That might mean that the file is "
                           "corrupted. That may also mean that you are using "
                           "a deprecated version of FFMPEG. On Ubuntu/Debian "
                           "for instance the version in the repos is deprecated. "
                           "Please update to a recent version from the website."

check the file 'moviepy/video/io/ffmpeg_reader.py line 135-140

        if not hasattr(self, 'lastread'):
            raise IOError(("MoviePy error: failed to read the first frame of "
                           "video file %s. That might mean that the file is "
                           "corrupted. That may also mean that you are using "
                           "a deprecated version of FFMPEG. On Ubuntu/Debian "
                           "for instance the version in the repos is deprecated. "
                           "Please update to a recent version from the website.")%(
                            self.filename))

maybe not hasattr(self, 'lastread') is the problem.

Is there another way to solve this problem?

[AaronNBrock]

What ended up happening with this?

[MaratZakirov]

For me this is work out of the box
ffmpeg -hwaccel cuvid -c:v h264_cuvid -i heavy_load.mp4 -c:v h264_nvenc -preset slow heavy_out.mp4
ffmpeg from anaconda

[iamtahirkhan6]

For me this is work out of the box
ffmpeg -hwaccel cuvid -c:v h264_cuvid -i heavy_load.mp4 -c:v h264_nvenc -preset slow heavy_out.mp4
ffmpeg from anaconda

How to use this with moviepy ?

[DavidM42]

I'm also interested if anyone found a way to enable ffmpeg gpu acceleration in moviepy?

[zhangdengp]

I have the same problem

[serbanc94]

Same issue here. If I modify the ffmpeg_reader.py file' cmd variable into that, I get the same error as above.
python cmd = ([get_setting("FFMPEG_BINARY")] + i_arg + ['-loglevel', 'error', '-f', 'image2pipe', '-vf', 'scale=%d:%d' % tuple(self.size), '-sws_flags', self.resize_algo, "-pix_fmt", self.pix_fmt, '-vcodec', 'rawvideo', '-hwaccel', 'cuvid', '-vcodec', 'h264_cuvid', '-c:v', 'h264_cuvid', '-preset','slow', '-'])

[zEdS15B3GCwq]

This is an old thread but seeing recent interest in it, I'll throw in my 2 cents. I'm assuming this is about video DECODING, not encoding. OP's post is confusing as it talks about decoding, the ffmpeg_reader file is modified, but the edits are on the encoding part of the ffmpeg command.

I can see several problems why it's not working for you:

1) the OP changed the output video codec (from rawvideo to nvenc), instead of changing the input codec. The line with rawvideo must stay there so that moviepy can read raw frames from the pipe.

2) I'm not an expert, but as far as I'm aware, input video codec specifications go before the input file: ffmpeg -c:v h264_cuvid -i input ... etc. In ffmpeg_reader.py, this means: ([get_setting("FFMPEG_BINARY"), '-c:v', 'h264_cuvid'] + i_arg + ... etc. This modification works.

2) Another probably issue is that if you use '-hwaccel cuvid', the decoded frame stays in GPU memory. My impression from taking a quick look at the code is that moviepy doesn't know how to deal with frame data on the GPU.

3) If you use an incorrect decoder (e.g. h264_cuvid for a hevc encoded video), moviepy won't be able to read the frames, and you get the same error code. Make sure you use the right decoder.

Example working code:
cmd = ([get_setting("FFMPEG_BINARY"), '-c:v', 'h264_cuvid'] + i_arg + ['-loglevel', 'error', '-f', 'image2pipe', '-vf', 'scale=%d:%d' % tuple(self.size), '-sws_flags', self.resize_algo, "-pix_fmt", self.pix_fmt, '-vcodec', 'rawvideo', '-'])

Before you spend a lot of energy on accelerating decoding, it's worth doing a few tests if it would even speed things up as you expect. I have mixed results with CPU vs GPU cuvid decoding; on some of my PCs the CPU decoding is faster, albeit it uses more CPU power obviously. The above way is not optimal in any case: I suspect that each frame is transferred about 3 times between the CPU and GPU (1 to GPU for decoding after loading, 2 to CPU into moviepy frame array, 3 to GPU to be displayed).

[hardyOne]

Interested in how to use GPU for mac
cmd = ([get_setting("FFMPEG_BINARY"), '-v:c', 'h264_videotoolbox'] + i_arg + ['-loglevel', 'error', '-f', 'image2pipe', '-vf', 'scale=%d:%d' % tuple(self.size), '-sws_flags', self.resize_algo, "-pix_fmt", self.pix_fmt, '-vcodec', 'rawvideo', '-'])

cmd = ([get_setting("FFMPEG_BINARY"), '-v:c', 'h265_videotoolbox'] + i_arg + ['-loglevel', 'error', '-f', 'image2pipe', '-vf', 'scale=%d:%d' % tuple(self.size), '-sws_flags', self.resize_algo, "-pix_fmt", self.pix_fmt, '-vcodec', 'rawvideo', '-'])

cmd = ([get_setting("FFMPEG_BINARY"), '-v:c', 'hevc_videotoolbox'] + i_arg + ['-loglevel', 'error', '-f', 'image2pipe', '-vf', 'scale=%d:%d' % tuple(self.size), '-sws_flags', self.resize_algo, "-pix_fmt", self.pix_fmt, '-vcodec', 'rawvideo', '-'])
do not work for me, and I got the following error
OSError: MoviePy error: failed to read the first frame of video file