Javacv: Rotate frames while using FFmpegFrameRecorder to stream to RTMP server

[tomerhatav]

Hello, I am using FFmpegFrameRecorder to stream video to my RTMP server.
Everything is working as expected but the frames are in landscape mode.
I am trying to rotate the using opencv methods below but the result is not good,
I get a purplish image.
Can anyone help?
Thanks in advance!

private void initRecorder() {
    //Video
    recorder = new FFmpegFrameRecorder(rtmplink, 320, 568, 2);
    recorder.setInterleaved(true);
    recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);
    recorder.setVideoBitrate(videoBitRate);
    recorder.setPixelFormat(avutil.AV_PIX_FMT_YUV420P);
    recorder.setFormat("flv");
    recorder.setVideoOption("preset", "veryfast");
    recorder.setVideoOption("tune", "zerolatency");
    recorder.setGopSize(GOP_LENGTH_IN_FRAMES);
    recorder.setAudioCodec(avcodec.AV_CODEC_ID_AAC);
    recorder.setSampleRate(sampleAudioRateInHz);
    recorder.setAudioBitrate(audioBitRate);
    recorder.setFrameRate(frameRate);
}

@Override
public void onPreviewFrame(byte[] data, Camera camera) {
    if (audioRecord == null || audioRecord.getRecordingState() != AudioRecord.RECORDSTATE_RECORDING) {
        startTime = System.currentTimeMillis();
        return;
    }

    if (recording) {
        if(frame == null){
            frame = new Frame(imageHeight, imageWidth, IPL_DEPTH_8U, 3);
        }

        yuvimage = IplImage.create(imageWidth, imageHeight * 3 / 2, IPL_DEPTH_8U, 1);
        yuvimage.getByteBuffer().put(data);

        rgbimage = IplImage.create(imageWidth, imageHeight, IPL_DEPTH_8U, 3);
        opencv_imgproc.cvCvtColor(yuvimage, rgbimage, opencv_imgproc.CV_YUV2BGR_NV21);

        if(!frontCamera){
            yuvimage=rotate(rgbimage,1);
        }else{
            yuvimage=rotate(rgbimage,-1);
        }

        long t = 1000 * (System.currentTimeMillis() - startTime);
        if (t > recorder.getTimestamp()) {
            recorder.setTimestamp(t);
        }
        ((ByteBuffer)frame.image[0].position(0)).put(yuvimage.getByteBuffer());
        try {
            recorder.record(frame);
        } catch (FFmpegFrameRecorder.Exception e) {
            e.printStackTrace();
        }
    }

    IplImage rotate(IplImage IplSrc,int angle) {
        IplImage img= IplImage.create(IplSrc.height(), IplSrc.width(), IplSrc.depth(), IplSrc.nChannels());
        cvTranspose(IplSrc, img);
        cvFlip(img, img, angle);
        return img;
    }
}

[tomerhatav]

This referenced issue might help me solve my problem? I don't understand.

[zouyongdang]

Did you solve your problem?

[saudet]

@tomerhatav No, the other way around.

[tomerhatav]

@zouyongdang No I didn't

[saudet]

Maybe your device doesn't support the standard Android format... What happens if you use CV_YUV2RGB_NV21 instead?

[tomerhatav]

I Have a nexus 5, I believe it's the most standard android there is. :)

[yangchch786]

Did you solve this problem? I have a same problem.

[tomerhatav]

Yes, I'm rotating directly the byte array received onPreviewFrame, without using the opencv functions (cvTranspose.cvFlip).

[saudet]

@tomerhatav Could you provide your code for the benefit of others? Or even better, integrate it in the RecordActivity sample and send me a PR? It would be awesome :)

[tomerhatav]

@saudet I'm using those functions here to rotate the byte[] from onPreviewFrame depends on the camera (back or front).
http://pastebin.com/Ps5Xkz4q

But be aware, it is using alot of cpu power so do not set camera preview size over 640x480.

[saudet]

Duplicate of https://github.com/bytedeco/javacv/issues/164. We can now use FFmpegFrameFilter to rotate more efficiently.

[jeka-uk]

Hi! everybody! Can somebody havs real work example streaming video from camera on Android to RTMP?
I spent more time and didn't find real work solution?

[y4nnick]

@jeka-uk you can find my solution under: https://github.com/bytedeco/javacv/issues/491, i will create a sample project from it soon