Javacv: Ubuntu 18.04 System FFmpegFrameGrabber.grab() Out of memory

[ml02218370]

I get the video stream for "Rtsp" and save it as an MP4 file.
After my test:
ubuntu System memory has been rising,but windows System memory normal

My code:

public class CameraPush {
    protected FFmpegFrameGrabber grabber = null;
    protected FFmpegFrameRecorder record = null;
    int width;
    int height;
    protected int audiocodecid;
    protected int codecid;
    protected double framerate;
    protected int bitrate;
    private int audioChannels;
    private int audioBitrate;
    private int sampleRate;

    public CameraPush() { }

    public CameraPush from() throws Exception {
        grabber = FFmpegFrameGrabberUtil.createDefault("rtsp://admin:[email protected]:554/id=1&type=0");
        grabber.setTimeout(5000);
        grabber.setOption("rtsp_transport", "tcp");
        grabber.setOption("stimeout", 5000000);
        try {
            grabber.start();
            width = grabber.getImageWidth();
            height = grabber.getImageHeight();         
            if (width == 0 && height == 0) {
                logger.showErrorLogger("[ERROR]   TimeOut...");
                grabber.stop();
                grabber.close();
                return null;
            }
            audiocodecid = grabber.getAudioCodec();
            codecid = grabber.getVideoCodec();
            framerate = grabber.getVideoFrameRate();
            bitrate = grabber.getVideoBitrate();
            audioChannels = grabber.getAudioChannels();
            audioBitrate = grabber.getAudioBitrate();
            if (audioBitrate < 1) {
                audioBitrate = 128 * 1000;
            }
        } catch (org.bytedeco.javacv.FrameGrabber.Exception e) {
            grabber.stop();
            grabber.close();
            return null;
        }
        return this;
    }

    public CameraPush toMp4(String savePath) throws Exception {
        record = new FFmpegFrameRecorder(savePath, grabber.getImageWidth(), grabber.getImageHeight(), grabber.getAudioChannels());
        record.setVideoCodec(avcodec.AV_CODEC_ID_H264);
        record.setGopSize((int)framerate/2);
        record.setFormat("mp4");
        record.setPixelFormat(avutil.AV_PIX_FMT_YUV420P);
        record.setFrameRate(grabber.getFrameRate());
        record.setSampleRate(grabber.getSampleRate());
        record.start();
        return this;
    }

    public CameraPush goMp4(Thread nowThread)
            throws org.bytedeco.javacv.FrameGrabber.Exception, org.bytedeco.javacv.FrameRecorder.Exception {
        long err_index = 0;
        int allFrameSize=10*(int)framerate; //save frame size
        int indexFrame=0;
        for (int no_frame_index = 0; no_frame_index < 5 && err_index < 5&&indexFrame<allFrameSize;) {
            try {
                nowThread.sleep(1);
                Frame captured_frame = null;
                if((captured_frame = grabber.grab()) != null){
                    record.setTimestamp(grabber.getTimestamp());
                    record.record(captured_frame);
                    indexFrame+=1;
                    err_index=0;
                }else {
                    err_index++;
                }
            } catch (InterruptedException e) {           
                break;
            } catch (org.bytedeco.javacv.FrameGrabber.Exception e) {
                err_index++;
            } catch (org.bytedeco.javacv.FrameRecorder.Exception e) {
                err_index++;
            }
        }
        record.stop();
        record.close();
        grabber.stop();
        grabber.close();
        System.gc();
        return this;
    }
}



public class CameraThread {
    public static class MyRunnable implements Runnable {
        public static ExecutorService es = Executors.newCachedThreadPool();
        private Thread nowThread;
        private String savePath;

        public MyRunnable(String savePath) {
            this.savePath = savePath;
        }

        public void setInterrupted() {
            nowThread.interrupt();
        }

        @Override
        public void run() {
            CameraPush push=null;
            try {
                nowThread = Thread.currentThread();
                push= new CameraPush(cameraPojo).from();
                if (push != null) {
                    push.toMp4(savePath).goMp4(nowThread); 
                }
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
    }
}

Test Code:

 Thread thread=new Thread(new Runnable() {
            @Override
            public void run() {
                int id=1;
                while (true) {
                      String savePath="/home/test/savefiles/"+String.valueOf(id)+".mp4";
                       CameraThread.MyRunnable job = new CameraThread.MyRunnable(savePath);
                       CameraThread.MyRunnable.es.execute(job);
                        id++;
                        try {
                            Thread.sleep(30000);
                        } catch (Exception ex) {
                        }               
                }
            }
        });
        thread.setDaemon(true);
        thread.start();

my pom:

<dependency>
            <groupId>org.bytedeco</groupId>
            <artifactId>javacv-platform</artifactId>
            <version>1.5.2</version>
 </dependency>

[saudet]

Please try with JavaCV 1.5.2.

[ml02218370]

Isn't it JavaCV 1.5.2?

org.bytedeco
javacv-platform
1.5.2

[ml02218370]

--
--org.bytedeco
--javacv-platform
--1.5.2
--

[saudet]

It might be related to "CameraPush". Have you tried without?

[ml02218370]

I just call grabber.grab() and the memory grows
I don't use "CameraPush"
Ubuntu system memory will also grow

[saudet]

Ok, please simplify your code! Thanks

[ml02218370]

Ok Each time I create a thread to execute the following method, the memory grows

      public void saveMp4(String savePath)  throws org.bytedeco.javacv.FrameGrabber.Exception, org.bytedeco.javacv.FrameRecorder.Exception {
        FFmpegFrameGrabber grabber = FFmpegFrameGrabber.createDefault("rtsp://admin:[email protected]:554/id=1&type=0");
        grabber.setTimeout(5000);
        grabber.setOption("rtsp_transport", "tcp");
        grabber.setOption("stimeout", "5000000");

        FFmpegFrameRecorder record = new FFmpegFrameRecorder(savePath, 1920, 1080, 0);
        record.setVideoCodec(avcodec.AV_CODEC_ID_H264);
        record.setGopSize(12);
        record.setFormat("mp4");
        record.setPixelFormat(avutil.AV_PIX_FMT_YUV420P);
        record.setFrameRate(25);

        grabber.start();
        record.start();
        long err_index = 0;
        int allFrameSize = 250; //save frame size
        int indexFrame = 0;
        for (int no_frame_index = 0; no_frame_index < 5 && err_index < 5 && indexFrame < allFrameSize; ) {
            try {
                Frame captured_frame = null;
                if ((captured_frame = grabber.grab()) != null) {
                    record.setTimestamp(grabber.getTimestamp());
                    record.record(captured_frame);
                    indexFrame += 1;
                    err_index = 0;
                } else {
                    err_index++;
                }
            } catch (org.bytedeco.javacv.FrameGrabber.Exception e) {
                err_index++;
            } catch (org.bytedeco.javacv.FrameRecorder.Exception e) {
                err_index++;
            }
        }
        record.stop();
        record.close();
        grabber.stop();
        grabber.close();
        System.gc();
      }   

[saudet]

Thank you! Does it only happen with RTSP or does it also happen with files?

[ml02218370]

I tested it,Ubuntu System running this program ".MP4" and ".FLV" also "out of memory"

[saudet]

Could you post here the whole stack trace that you get with the exception?

[saudet]

BTW, it looks like you're only converting files. It's probably easier to just use the ffmpeg program:
http://bytedeco.org/javacpp-presets/ffmpeg/apidocs/org/bytedeco/ffmpeg/ffmpeg.html

[saudet]

I've added PointerScope inside the methods of FFmpegFrameGrabber and FFmpegFrameRecorder in commit https://github.com/bytedeco/javacv/commit/28b90ef25cc54d4deeba89f078270793abdd525f. Please give it a try with the snapshots: http://bytedeco.org/builds/

[canelzio]

I think I'm facing an issue related to this one. When trying to transcode an audio file from AMR (Narrow Band) to WAV in G711 codec I got an OOM error.

Following a snipped of my code (in my case it is embedded in a WebApp):

public String transcodeAMR2WAV(File inFile) throws Exception {
        FFmpegFrameGrabber grabber    = null;
        FFmpegFrameRecorder recorder  = null;

        String outFile = null;

       try {
            grabber = new FFmpegFrameGrabber(inFile);
            grabber.start();

            int outBitRate = grabber.getAudioBitrate();
            outFile = inFile.getAbsolutePath() + ".wav";

            recorder = new FFmpegFrameRecorder(outFile, grabber.getAudioChannels());
            recorder.setSampleRate(8000);
            recorder.setFormat("wav");
            recorder.setAudioCodec(org.bytedeco.ffmpeg.global.avcodec.AV_CODEC_ID_PCM_ALAW);

            if(outBitRate > 0) {
                recorder.setAudioBitrate(outBitRate);
            }

            recorder.start();

            Frame frame = null;
            while ((frame = grabber.grabSamples()) != null) {
                recorder.record(frame);

            }

        } finally {
            try { if(recorder != null) { recorder.stop(); } } catch (Exception ignore) { }
            try { if(recorder != null) { recorder.release(); } } catch (Exception ignore) { }
            try { if(recorder != null) { recorder.close(); } } catch (Exception ignore) { } // Calling close() method should not have effect since the object has been already stopped and released
            recorder = null;

            try { if(grabber != null) { grabber.stop(); } } catch (Exception ignore) { }
            try { if(grabber != null) { grabber.release(); } } catch (Exception ignore) { }
            try { if(grabber != null) { grabber.close(); } } catch (Exception ignore) { } // Calling close() method should not have effect since the object has been already stopped and released
            grabber = null;

        }

    return outFile;

}

Anyway I was able to reproduce the issue by repeatedly submitting the same transcoding request in a while(true) cycle, something like this:

#!/bin/bash

while true; do
        curl -F "format=G711" -F "uploadedFile=@/tmp/infile.amr" "http://localhost:8686/MyApp/convert" > /tmp/outfile.wav
done

Let me know if you need anything else.

Thank you.

[saudet]

I think I've fixed all the memory leaks occurring in FFmpegFrameGrabber and FFmpegFrameRecorder.
Please give it a try with the snapshots: http://bytedeco.org/builds/

I'm not noticing anything suspicious anymore when running these lines for over an hour:

        while (true) {
            saveMp4("somevideo.mp4");
            transcodeAMR2WAV(new File("someaudio.mp3"));
            System.gc();
            System.out.println(Pointer.formatBytes(Pointer.physicalBytes()));
        }

Physical memory usage does appear to rise a bit still, but I haven't been able to find a cause, so it might be happening because of memory fragmentation from FFmpeg itself... In any case, please let me know if it still throws OutOfMemoryError. Thanks!

[canelzio]

I think you hit the point Samuel!
I started the snapshot test session more than 7 hours ago and after an initial growing (from 5.2% to 7.3% occupancy) memory stabilized and does not seem to grow anymore.
Moreover, I did not need to use the PointerScope in my code.
Thank you.

[saudet]

Fixes included in JavaCV 1.5.4. Enjoy! Thanks for reporting and testing this out