Javacv: Crash Java using 1.5.4 on Mac with libx264

[Wada-H]

There seems to be a problem with the FFmpegFrameRecorder that worked fine from 1.5.1 to 1.5.3.
It seems that there is no problem if the codec is mpeg4, but when I use h264, Java will be crashed.
I would appreciate it if you could let me know if there is a workaround.

A fatal error has been detected by the Java Runtime Environment:
SIGSEGV (0xb) at pc=0x00000001c7c9ed92, pid=6835, tid=0x000000000001e60f

JRE version: Java(TM) SE Runtime Environment (8.0_172-b11) (build 1.8.0_172-b11)
Java VM: Java HotSpot(TM) 64-Bit Server VM (25.172-b11 mixed mode bsd-amd64 compressed oops)
Problematic frame:
C [libavcodec.58.dylib+0xfc2d92] aver_isf_history+0x2ae3a2

[saudet]

Please provide some code and data to reproduce this.

[Wada-H]

Thank you for your reply.
I extracted a part of the code.
When I checked where it was crashing, it seems to be the "recorder.recode(f_img)" part.
*

FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(dir + file, stack_img.getWidth(),stack_img.getHeight());


recorder.setVideoCodec(codec); //AV_CODEC_ID_H264
recorder.setFormat(ct);
recorder.setPixelFormat(AV_PIX_FMT_YUV420P);
recorder.setFrameRate(fps);
recorder.setVideoBitrate(br);

Java2DFrameConverter j2dfc = new Java2DFrameConverter();

try {
    recorder.start();
    for(int i = 1; i <= stack_img.getSize(); i++){
        IJ.showStatus("Encording movie...");
        IJ.showProgress(i+1, t);
        String time_str = conv_sec_to_time(((i - zeroposition)*interval));
        if(fixMin){
            time_str = convertSecToMin(((i+1 - zeroposition)*interval));
        }
        ip = stack_img.getProcessor(i);
        ip.setFont(time_stamp_f);
        ip.setColor(Color.white);

        if(scale == true){
            ip.setAntialiasedText(false);
            ip.fill(scale_roi);
        }

        if(time_stamp == true){
            ip.setAntialiasedText(true);
            ip.drawString(time_str, offset_t_s_x, offset_t_s_y); //time stamp
        }

        buffered_img = ip.getBufferedImage();

        org.bytedeco.javacv.Frame f_img = j2dfc.convert(conv_ARGB_to_RGBA(buffered_img));
        recorder.record(f_img); //crash point
        buffered_img.flush();
    }
    recorder.stop();

}catch(FrameRecorder.Exception err){
        IJ.log("error:" + err);
}

[saudet]

Thanks! Can you reduce that to ~10 lines for code and provide a sample image that fails?

[Wada-H]

I've taken some parts that it don't need, but it's difficult to fit in 10 lines.

public void testrecord(){

    f_stack_img = getFlattenRGB(imp); //Convert the loaded image to RGB
    stack_img = this.expandForMovie(f_stack_img); //Adjusted to a multiple of 16 in height and width

    ImageProcessor  ip;
    BufferedImage buffered_img;

    FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(dir + file, stack_img.getWidth(), stack_img.getHeight());

    recorder.setVideoCodec(codec); //AV_CODEC_ID_H264
    recorder.setFormat(ct); //mov
    recorder.setPixelFormat(AV_PIX_FMT_YUV420P);
    recorder.setFrameRate(fps);
    recorder.setVideoBitrate(br);

    Java2DFrameConverter j2dfc = new Java2DFrameConverter();

    try {
        recorder.start();
        for(int i = 1; i <= stack_img.getSize(); i++){
            ip = stack_img.getProcessor(i);
            buffered_img = ip.getBufferedImage();
            org.bytedeco.javacv.Frame f_img = j2dfc.convert(conv_ARGB_to_RGBA(buffered_img));
            recorder.record(f_img); //crash point
            buffered_img.flush();
        }
        recorder.stop();
    }catch(FrameRecorder.Exception err){
        IJ.log("error:" + err);
    }

}

HyperStack.tif.zip

[saudet]

It happens only when you use it with the rest of your code? It doesn't happen only with FFmpegFrameRecorder?

[Wada-H]

*The expression is immature because English is not my mother tongue, but it is not malicious.
I don't understand the meaning of the question.
How can it be used alone?
Isn't using javacv as a library to use it as own code?
It doesn't make any sense if it works only with FFmpegFrameRecorder even though it is supposed to be used as a library.
On the contrary, please tell me the code that could convert my attached image to video with h264 codec in the mov container in that environment.
Once I know that, I can look for any mistakes in my code.
But how do you explain that this code works fine with just the codec changed to mpeg4?

[saudet]

Your English is fine, don't worry! Your code looks fine, but I cannot test it because some functions are missing. Because of that, I assume that FFmpegFrameRecorder works fine when used outside your application, am I correct?

[Wada-H]

Thank you for my English,
To be honest, I've never used it outside the application, so I need to think about how to test it.
And I think this is the case with each other, but I'm simply skeptical about the h264 codec.
Anyway, just using my image to tell me the code that works well with the h264 codec may help me find a solution, so I'd be happy if you could show me that code.

[saudet]

The tests here encode in H.264 and work just fine:
https://github.com/bytedeco/javacv/tree/master/platform/src/test/java/org/bytedeco/javacv

[Wada-H]

Thank you for your information.
The same happens with the code below.(mpeg4 is ok, h264 is same problem)
You should be able to try this code in your environment as well.
*Java version, 1.8_172, 1.8_202, adopt1.8_212, 11.0.2, and 12.0.1 will be crashed.

import org.bytedeco.javacv.FFmpegFrameRecorder;

import org.bytedeco.javacv.FrameRecorder;
import org.bytedeco.javacv.Java2DFrameConverter;

import javax.imageio.ImageIO;
import javax.imageio.ImageReader;
import javax.imageio.stream.ImageInputStream;
import java.awt.image.BufferedImage;
import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Iterator;

import static org.bytedeco.ffmpeg.global.avutil.AV_PIX_FMT_YUV420P;
import static org.bytedeco.ffmpeg.global.avcodec.AV_CODEC_ID_H264;
import static org.bytedeco.ffmpeg.global.avcodec.AV_CODEC_ID_MPEG4;

public class Main {


    public static void main(String[] args) {


        BufferedImage buffered_img;
        Iterator readers = ImageIO.getImageReadersByFormatName("tif");
        ImageReader reader = (ImageReader) readers.next();

        // TIFF読み込み
        String dir = "/Users/hwada/Desktop/java_ij/forJavaCV/";
        String file = "HyperStack.tif";
        File f = new File(dir + file);
        ImageInputStream iis = null;

        try {
            iis = ImageIO.createImageInputStream(f);
        } catch (IOException e) {
            e.printStackTrace();
        }

        reader.setInput(iis, true);
        ArrayList<BufferedImage> imageList = new ArrayList();
        for (int i = 0;; i++) {
            try {
                imageList.add(reader.read(i));

            } catch (IndexOutOfBoundsException ioe) {
                break;
            } catch (IOException e) {
                e.printStackTrace();
            }
        }

        //read multi tiff is O.K


        int width = imageList.get(0).getWidth();
        int height = imageList.get(0).getHeight();

        String saveFile = "test.mov";
        FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(dir + saveFile, width, height);

        recorder.setVideoCodec(AV_CODEC_ID_H264);
        //recorder.setVideoCodec(AV_CODEC_ID_MPEG4);

        recorder.setFormat("mov");
        recorder.setPixelFormat(AV_PIX_FMT_YUV420P);
        recorder.setFrameRate(30);
        recorder.setVideoBitrate(5000); //sample codeでは使われていない

        Java2DFrameConverter j2dfc = new Java2DFrameConverter();

        try {
            recorder.start();
            for(int i = 0; i < imageList.size(); i++){
                buffered_img = imageList.get(i);
                org.bytedeco.javacv.Frame f_img = j2dfc.convert(buffered_img);
                recorder.record(f_img); //crash point
                buffered_img.flush();
            }
            recorder.stop();
        }catch(FrameRecorder.Exception err){
            //どっかにログ出力がいると思われる
        }


    }



}

[saudet]

Thank you! I am able to reproduce this crash on Mac, but not on Linux. Indeed, it looks like the latest version of libx264 has issues on Mac. You might want to report this upstream: https://www.videolan.org/developers/x264.html

In the meantime, OpenH264 doesn't seem to have any issues on Mac, so you could use that codec if that one is OK as well:

recorder.setVideoCodecName("libopenh264");

[Wada-H]

Thank you for reporting the workaround and confirmation.
I will try the way and report the issue to x264 site.

[saudet]

If you need libx264, I was able to work around this by disabling assembly optimizations.
Please give it a try with JavaCV 1.5.5-SNAPSHOT: http://bytedeco.org/builds/

[saudet]

The workaround has been released with JavaCV 1.5.5, but it would still be nice to figure out how we could use the assembly optimizations even on Mac. In any case, thanks for reporting this problem!