Javacv: A problem with the setTimeout method of FFmpegFrameGrabber

[cndota123]

I found that after I used the setTimeout method of FFmpegFrameGrabber, the program is running a frequent crash, and it appears at any stage, including start, grab, stop.

[saudet]

That property isn't currently even used by FFmpegFrameGrabber, so it can't be the cause of the crashes.

[cndota123]

This is my code

public class Stream {

    private static Logger logger = LoggerFactory.getLogger(Stream.class);

    private FFmpegFrameGrabber grabber = null;
    private FFmpegFrameRecorder recorder = null;
    private int width;
    private int height;
    private String sourceUrl;
    private String outputUrl;
    private int videoBitrate;
    private boolean isRun = false;
    private int restartTime = 0;


    public int getInterrupted() {
        return interrupted;
    }

    public void setInterrupted(int interrupted) {
        this.interrupted = interrupted;
    }

    private volatile int interrupted = 0;

    private int audioRecord;


    public String getOutputUrl() {
        return outputUrl;
    }

    public Stream(String sourceUrl, String outputUrl, int width, int height, int audioRecord, int videoBitrate) {
        this.width = width;
        this.height = height;
        this.sourceUrl = sourceUrl;
        this.outputUrl = outputUrl;
        this.audioRecord = audioRecord;
        this.videoBitrate = videoBitrate;
    }

    public Stream pull() throws CommonException {
        grabber = new FFmpegFrameGrabber(sourceUrl);
        grabber.setOption("rtsp_transport", "tcp");
        grabber.setAudioStream(Integer.MAX_VALUE);
        //grabber.setTimeout(3 * 1000000);
        avutil.av_log_set_level(avutil.AV_LOG_ERROR);
        int pullTime = 0;
        for (; pullTime <= 3; pullTime++) {
            try {
                logger.info("try pull from " + sourceUrl);
                grabber.start();
                break;
            } catch (FrameGrabber.Exception e) {
                logger.error(sourceUrl + " pull fail,try pull again");
                e.printStackTrace();
                try {
                    grabber.stop();
                } catch (FrameGrabber.Exception e1) {
                    e1.printStackTrace();
                    logger.error(sourceUrl + " pull fail and grabber stop failed in restart");
                }
                if (pullTime == 3)
                    throw new CommonException("2001", Result.getMessage("2001"));
            }
        }
        logger.info("pull success from " + sourceUrl);
        return this;
    }

    public Stream push() throws CommonException {
        recorder = new FFmpegFrameRecorder(outputUrl, width, height, audioRecord);
        recorder.setFormat("flv");
        recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);
        //recorder.setAudioCodecName("acc");
        //recorder.setAudioChannels(audioRecord);
        //recorder.setSampleRate(44100);
        //recorder.setVideoBitrate(videoBitrate);
        recorder.setVideoOption("vprofile", "baseline");
        recorder.setVideoOption("tune", "zerolatency");
        recorder.setFrameRate(25);
        recorder.setGopSize(50);
        int pushTime = 0;
        for (; pushTime <= 3; pushTime++) {
            try {
                logger.info("try push to " + outputUrl);
                recorder.start();
                break;
            } catch (FrameRecorder.Exception e) {
                logger.error(outputUrl + " push fail");
                e.printStackTrace();
                try {
                    recorder.stop();
                } catch (FrameRecorder.Exception e1) {
                    e1.printStackTrace();
                    logger.error(outputUrl + " push fail and recorder stop failed in restart");
                }
                if (pushTime == 3)
                    throw new CommonException("2002", Result.getMessage("2002"));
            }
        }
        logger.info("push success to " + outputUrl);
        return this;
    }

    public void start() throws CommonException {
        long startTime = 0;
        int errorIndex = 0;
        Frame frame;
        isRun = true;
        for (int noFrameIndex = 0; interrupted == 0 && (noFrameIndex < 5 || errorIndex > 1); ) {
            try {
                if (startTime == 0)
                    startTime = System.currentTimeMillis();
                long time = System.currentTimeMillis() - startTime;
                recorder.setTimestamp(1000 * time);
                frame = grabber.grab();
                if (frame == null || frame.image == null) {
                    noFrameIndex++;
                    continue;
                }
                recorder.record(frame);
                noFrameIndex = 0;
            } catch (FrameRecorder.Exception e) {
                e.printStackTrace();
                errorIndex++;
            } catch (FrameGrabber.Exception e) {
                e.printStackTrace();
                errorIndex++;
            }
        }
        if (interrupted == 0 && restartTime < 100)
            restart();
        else
            stop();
    }


    public void stop() {
        try {
            logger.info(outputUrl + " is closing");
            grabber.stop();
            recorder.stop();
            isRun = false;
            logger.info(outputUrl + " has been closed");
        } catch (FrameGrabber.Exception e) {
            logger.error(sourceUrl + " pull stream close failed");
            e.printStackTrace();
        } catch (FrameRecorder.Exception e) {
            logger.error(outputUrl + " push stream close failed");
            e.printStackTrace();
        }

    }

    public void restart() throws CommonException {
        logger.info(outputUrl + "is restarting");
        stop();
        restartTime++;
        this.pull().push().start();
    }

    public boolean isRun() {
        return isRun;
    }
}

When I enable this code

grabber.setTimeout(3 * 1000000);

Java crash occurred four times a day, and the program could not run for a long time.
Here is some information about java crash.

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00007ffaf26d3b46, pid=892, tid=0x0000000000002de4
#
# JRE version: Java(TM) SE Runtime Environment (8.0_102-b14) (build 1.8.0_102-b14)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.102-b14 mixed mode windows-amd64 compressed oops)
# Problematic frame:
# C  [avformat-58.dll+0x43b46]
#
# Core dump written. Default location: D:\streamTest\hs_err_pid892.mdmp
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
...
Stack: [0x0000000032e20000,0x0000000032f20000],  sp=0x0000000032f1ebf0,  free space=1018k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j  org.bytedeco.javacpp.avformat.av_dump_format(Lorg/bytedeco/javacpp/avformat$AVFormatContext;ILjava/lang/String;I)V+0
j  org.bytedeco.javacv.FFmpegFrameGrabber.startUnsafe()V+787
j  org.bytedeco.javacv.FFmpegFrameGrabber.start()V+1
j  com.zzx.smarttown.stream.Stream.pull()Lcom/zzx/smarttown/stream/Stream;+67
j  com.zzx.smarttown.stream.StreamOperate$1.run()V+4
j  java.lang.Thread.run()V+11
v  ~StubRoutines::call_stub

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00007ffaf2958b83, pid=9572, tid=0x00000000000013e0
#
# JRE version: Java(TM) SE Runtime Environment (8.0_102-b14) (build 1.8.0_102-b14)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.102-b14 mixed mode windows-amd64 compressed oops)
# Problematic frame:
# C  [avutil-56.dll+0x28b83]
#
# Core dump written. Default location: D:\streamTest\hs_err_pid9572.mdmp
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
...
Stack: [0x000000002af80000,0x000000002b080000],  sp=0x000000002b07cec8,  free space=1011k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [avutil-56.dll+0x28b83]
C  [avcodec-58.dll+0x28ec30]
C  [avcodec-58.dll+0x50803c]
C  [avformat-58.dll+0x16b4f0]
C  [avformat-58.dll+0x16c415]
C  [avformat-58.dll+0x17907b]
C  [jniavformat.dll+0x2e4b6]
C  0x0000000002487f54

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j  org.bytedeco.javacpp.avformat.avformat_close_input(Lorg/bytedeco/javacpp/avformat$AVFormatContext;)V+0
j  org.bytedeco.javacv.FFmpegFrameGrabber.releaseUnsafe()V+202
j  org.bytedeco.javacv.FFmpegFrameGrabber.release()V+1
j  org.bytedeco.javacv.FFmpegFrameGrabber.stop()V+1
j  com.zzx.smarttown.stream.Stream.stop()V+34
j  com.zzx.smarttown.stream.Stream.restart()V+31
j  com.zzx.smarttown.stream.StreamOperate.getLiveUrl(Ljava/lang/String;Ljava/lang/String;)Ljava/lang/String;+213
j  com.zzx.smarttown.service.StreamService.getLiveStream(Ljava/lang/String;Ljava/lang/String;)Ljava/lang/String;+6
j  com.zzx.smarttown.controller.StreamController.getLiveStream(Ljava/lang/String;Ljava/lang/String;)Lcom/zzx/smarttown/exception/Result;+13
j  sun.reflect.GeneratedMethodAccessor32.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+48
J 3926 C2 java.lang.reflect.Method.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (62 bytes) @ 0x0000000003449cc8 [0x0000000003449c20+0xa8]
j  org.springframework.web.method.support.InvocableHandlerMethod.doInvoke([Ljava/lang/Object;)Ljava/lang/Object;+16
j  org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(Lorg/springframework/web/context/request/NativeWebRequest;Lorg/springframework/web/method/support/ModelAndViewContainer;[Ljava/lang/Object;)Ljava/lang/Object;+75
j  org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(Lorg/springframework/web/context/request/ServletWebRequest;Lorg/springframework/web/method/support/ModelAndViewContainer;[Ljava/lang/Object;)V+4
j  org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Lorg/springframework/web/method/HandlerMethod;)Lorg/springframework/web/servlet/ModelAndView;+276
j  org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Lorg/springframework/web/method/HandlerMethod;)Lorg/springframework/web/servlet/ModelAndView;+81
j  org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljava/lang/Object;)Lorg/springframework/web/servlet/ModelAndView;+7
j  org.springframework.web.servlet.DispatcherServlet.doDispatch(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+310
j  org.springframework.web.servlet.DispatcherServlet.doService(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+308
j  org.springframework.web.servlet.FrameworkServlet.processRequest(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+71
j  org.springframework.web.servlet.FrameworkServlet.doGet(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+3
j  javax.servlet.http.HttpServlet.service(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+35
j  org.springframework.web.servlet.FrameworkServlet.service(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+33
j  javax.servlet.http.HttpServlet.service(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V+30
J 5491 C1 org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (388 bytes) @ 0x00000000032c968c [0x00000000032c8ba0+0xaec]
J 5490 C1 org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (105 bytes) @ 0x0000000002d69a4c [0x0000000002d699c0+0x8c]
j  org.apache.tomcat.websocket.server.WsFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+21
J 5491 C1 org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (388 bytes) @ 0x00000000032c9134 [0x00000000032c8ba0+0x594]
J 5490 C1 org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (105 bytes) @ 0x0000000002d69a4c [0x0000000002d699c0+0x8c]
j  com.zzx.smarttown.config.CorsFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+80
J 5491 C1 org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (388 bytes) @ 0x00000000032c9134 [0x00000000032c8ba0+0x594]
J 5490 C1 org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (105 bytes) @ 0x0000000002d69a4c [0x0000000002d699c0+0x8c]
j  org.springframework.web.filter.RequestContextFilter.doFilterInternal(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/FilterChain;)V+21
j  org.springframework.web.filter.OncePerRequestFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+111
J 5491 C1 org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (388 bytes) @ 0x00000000032c9134 [0x00000000032c8ba0+0x594]
J 5490 C1 org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (105 bytes) @ 0x0000000002d69a4c [0x0000000002d699c0+0x8c]
j  org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/FilterChain;)V+95
j  org.springframework.web.filter.OncePerRequestFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+111
J 5491 C1 org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (388 bytes) @ 0x00000000032c9134 [0x00000000032c8ba0+0x594]
J 5490 C1 org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (105 bytes) @ 0x0000000002d69a4c [0x0000000002d699c0+0x8c]
j  org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/FilterChain;)V+87
j  org.springframework.web.filter.OncePerRequestFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+111
J 5491 C1 org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (388 bytes) @ 0x00000000032c9134 [0x00000000032c8ba0+0x594]
J 5490 C1 org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (105 bytes) @ 0x0000000002d69a4c [0x0000000002d699c0+0x8c]
j  org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/FilterChain;)V+53
j  org.springframework.web.filter.OncePerRequestFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V+111
J 5491 C1 org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (388 bytes) @ 0x00000000032c9134 [0x00000000032c8ba0+0x594]
J 5490 C1 org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (105 bytes) @ 0x0000000002d69a4c [0x0000000002d699c0+0x8c]
j  org.apache.catalina.core.StandardWrapperValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+688
j  org.apache.catalina.core.StandardContextValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+166
j  org.apache.catalina.authenticator.AuthenticatorBase.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+260
j  org.apache.catalina.core.StandardHostValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+138
j  org.apache.catalina.valves.ErrorReportValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+6
j  org.apache.catalina.core.StandardEngineValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V+71
j  org.apache.catalina.connector.CoyoteAdapter.service(Lorg/apache/coyote/Request;Lorg/apache/coyote/Response;)V+199
j  org.apache.coyote.http11.Http11Processor.service(Lorg/apache/tomcat/util/net/SocketWrapperBase;)Lorg/apache/tomcat/util/net/AbstractEndpoint$Handler$SocketState;+806
j  org.apache.coyote.AbstractProcessorLight.process(Lorg/apache/tomcat/util/net/SocketWrapperBase;Lorg/apache/tomcat/util/net/SocketEvent;)Lorg/apache/tomcat/util/net/AbstractEndpoint$Handler$SocketState;+113
j  org.apache.coyote.AbstractProtocol$ConnectionHandler.process(Lorg/apache/tomcat/util/net/SocketWrapperBase;Lorg/apache/tomcat/util/net/SocketEvent;)Lorg/apache/tomcat/util/net/AbstractEndpoint$Handler$SocketState;+414
j  org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun()V+191
j  org.apache.tomcat.util.net.SocketProcessorBase.run()V+21
j  java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V+95
j  java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5
j  org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run()V+4
j  java.lang.Thread.run()V+11
v  ~StubRoutines::call_stub

This is two of them.I used google search to find a similar error about setting the timeout result. Although his problem is different from mine, I think I should try it, and it works successfully.

[saudet]

Like I said, this is not related to setTimeout(). It might be related to some of the issues with threads fixed in JavaCV 1.5 though. Could you please try again with that version? Also try to call the following:

        grabber.setOption("threads", "1");
        grabber.setVideoOption("threads", "1");
        grabber.setAudioOption("threads", "1");
        recorder.setOption("threads", "1");
        recorder.setVideoOption("threads", "1");
        recorder.setAudioOption("threads", "1");

That often helps to avoid threading issues in many codecs.

[cndota123]

The version I'm using is 1.4.4,I will test the program according to your instructions.

[cndota123]

After switching the version to 1.5, no similar error occurred. Adding the configuration is the same, but the previous version of the program does not have the same error again during the running, so I think this may be related to the running environment or the running status of the program at the time. Related, I will try to reproduce this problem, which may take a long time to test.

[saudet]

Good! It's probably something that's been fixed upstream.