Javacv: Audio and video are out of sync still appear when i change the recorder frame rate(I use a fixed frame rate)
the base code is:
FFmpegFrameGrabber frameGrabber = new FFmpegFrameGrabber(path);
String fileName = null;
Frame captured_frame = null;
FFmpegFrameRecorder recorder = null;
try {
frameGrabber.start();
fileName = "/Users/mac/Desktop/B612/2bc/"+System.currentTimeMillis()+".mp4";
int height = frameGrabber.getImageHeight();
int widht = frameGrabber.getImageWidth();
recorder = new FFmpegFrameRecorder(fileName, widht, height, frameGrabber.getAudioChannels());
recorder.setFrameRate(frameGrabber.getFrameRate());
recorder.setSampleRate(frameGrabber.getSampleRate());
recorder.setAudioChannels(2);
recorder.setVideoOption("preset", "faster");
recorder.setVideoOption("strict", "experimental");
recorder.setOption("movflags", " faststart");
// yuv420p,像素
recorder.setPixelFormat(avutil.AV_PIX_FMT_YUV420P);
recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);
recorder.setAudioCodec(avcodec.AV_CODEC_ID_AAC);
recorder.setFormat("mp4");
//比特
recorder.setVideoBitrate(VIDEO_BITRATE);
recorder.start();
while (true) {
try {
captured_frame = frameGrabber.grabFrame();
if (captured_frame == null) {
System.out.println("!!! end cvQueryFrame");
break;
}
recorder.setTimestamp(frameGrabber.getTimestamp());
recorder.record(captured_frame);
} catch (Exception e) {
}
}
recorder.stop();
recorder.release();
frameGrabber.stop();
} catch (Exception e) {
e.printStackTrace();
}
If I don't change the frame rate，I use the frame rate of the original video directly and don't record the time stamp,like that:
recorder.setFrameRate(frameGrabber.getFrameRate());
and remove
recorder.setTimestamp(frameGrabber.getTimestamp());
The result is OK, and there will be no asynchronous phenomenon...
But,If I change to a fixed frame rate, the frame rate is not the same as the original video frame rate,like that:
recorder.setFrameRate(30);
The following exceptions will occur
[mp4 @ 0x7ff75798ce00] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 58368 >= 58368
org.bytedeco.javacv.FrameRecorder$Exception: av_interleaved_write_frame() error -22 while writing interleaved video packet.
at org.bytedeco.javacv.FFmpegFrameRecorder.writePacket(FFmpegFrameRecorder.java:1236)
at org.bytedeco.javacv.FFmpegFrameRecorder.recordImage(FFmpegFrameRecorder.java:1031)
at org.bytedeco.javacv.FFmpegFrameRecorder.record(FFmpegFrameRecorder.java:918)
at org.bytedeco.javacv.FFmpegFrameRecorder.record(FFmpegFrameRecorder.java:911)
at com.b612kaji.h5.api.TestMain.convert(TestMain.java:507)
at com.b612kaji.h5.api.TestMain.main(TestMain.java:75)
And If I add the following code，Because I looked through similar questions and saw that it was suggested to write like this：
long gt = frameGrabber.getTimestamp();
long rt = recorder.getTimestamp();
if(gt > rt){
recorder.setTimestamp(gt);
}
The above abnormal information will not appear, but the audio and video will be out of sync. The picture is obviously delayed, and the first frame seems to have a black screen.
I saw your other articles that suggested FFmpegFrameFilter be used to set the fixed frame rate, but I used it and it didn't work.
I found that the time stamp of the next frame of the original video is indeed smaller than that of the previous frame, so I don't know how to solve the problem. I can directly use the command of ffmpeg, which is OK.The ffmpeg command is as follows：
17:01:11.667 [main] INFO net.bramp.ffmpeg.RunProcessFunction - /usr/local/bin/ffmpeg -y -v error -i media_1575022321747.MP4 -pass 1 -passlogfile da2c3f25-3ef4-44c7-92da-e7f279f9f0f6 -strict experimental -f mp4 -vcodec libx264 -movflags faststart -r 25/1 -b:v 1000000 -an -sn /dev/null
17:01:23.056 [main] INFO net.bramp.ffmpeg.RunProcessFunction - /usr/local/bin/ffmpeg -y -v error -i media_1575022321747.MP4 -pass 2 -passlogfile da2c3f25-3ef4-44c7-92da-e7f279f9f0f6 -strict experimental -f mp4 -vcodec libx264 -movflags faststart -r 25/1 -b:v 1000000 -acodec aac -ac 1 -ar 48000 -b:a 32768 -sn /Users/mac/Desktop/B612/ms/mp4/out/abcde.mp4
I want to know if there is any solution with javacv to change the frame rate and ensure the synchronization of audio and video，There's no way...
GeneralLHW
All 29 comments
If you change the frame rate, you'll also need to change the number of frames accordingly.
saudet
on 10 Apr 2020
If you change the frame rate, you'll also need to change the number of frames accordingly.
thank you for replay..
you mean:
For example, I have a video with a frame rate of 25 and a duration of 10 seconds. That's 250 frames. I want to change the frame rate to 30, which should be 300 frames at this time. But what should I do for the 50 frames that are different? What to do in the code?
GeneralLHW
on 10 Apr 2020
You'll need to add 50 frames, yes, that's the idea.
saudet
on 10 Apr 2020
You'll need to add 50 frames, yes, that's the idea.
So how can I get the 50 frames, how can I distribute them evenly, and how can I keep the length of the video unchanged?
GeneralLHW
on 10 Apr 2020
You could use the fps filter for that. The code from issue #717 should work now.
saudet
on 10 Apr 2020
Duplicate of #717
saudet
on 10 Apr 2020
You could use the
fpsfilter for that. The code from issue #717 should work now.
Still doesn't work...So is there a good way?
GeneralLHW
on 11 Apr 2020
I'm pretty sure it works, but we'll have to see if anyone is willing to take the time to figure this out.
saudet
on 11 Apr 2020
我敢肯定它能奏效，但我们必须看看是否有人愿意花时间来解决这个问题。
This is what I wrote, but the frame rate of the final video does not change. It's still the frame rate of the original video：
FFmpegFrameFilter fFmpegFrameFilter = new
FFmpegFrameFilter("fps=fps=25",frameGrabber.getImageWidth(), frameGrabber.getImageHeight());
Frame capturedFrame;
while (true) {
try {
capturedFrame = frameGrabber.grabFrame();
if (capturedFrame == null) {
log.warn("!!! end cvQueryFrame..video:{}",path);
break;
}
if (capturedFrame.image != null) {
fFmpegFrameFilter.push(capturedFrame);
if (fFmpegFrameFilter.pull() != null) {
recorder.record(capturedFrame);
}
}
} catch (Exception e) {
}
}
Check for any warning or error message in the log displayed on the console.
saudet
on 11 Apr 2020
Check for any warning or error message in the log displayed on the console.
I checked that there was no error message.
I found that after I used ffmpegframefilter, the generated video had no sound. I was wondering why?
GeneralLHW
on 13 Apr 2020
If you don't see any messages on the console, make sure FFmpegLogCallback.set() has been called.
saudet
on 13 Apr 2020
If you don't see any messages on the console, make sure
FFmpegLogCallback.set()has been called.
Normal information is output：
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'https://abc.com/media_1553508014839.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
title : B612咔叽
encoder : Lavf57.71.100
genre : {"version":"8.1.1","recordTime":"11","sections":"(1, 1)","frames":330,"sectionId":28}
Duration: 00:00:11.21, start: 0.000000, bitrate: 5382 kb/s
Stream #0:0(eng): Video: h264 (Baseline) (avc1 / 0x31637661), yuv420p(tv, smpte170m/bt470bg/smpte170m), 720x1280, 5247 kb/s, SAR 1:1 DAR 9:16, 29.89 fps, 90k tbr, 90k tbn, 180k tbc (default)
Metadata:
handler_name : VideoHandler
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default)
Metadata:
handler_name : SoundHandler
12:35:57.191 [main] INFO com.b612kaji.h5.api.TestMain - wight:720,height:1280,fileName:/Users/mac/Desktop/B612/2bc/1586752557190.mp4
[libx264 @ 0x7fcf6c31e600] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x7fcf6c31e600] profile High, level 3.1, 4:2:0, 8-bit
[libx264 @ 0x7fcf6c31e600] 264 - core 157 - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=1 ref=2 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=4 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=12 lookahead_threads=3 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=1 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=20 rc=abr mbtree=1 bitrate=1378 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to '/Users/mac/Desktop/B612/2bc/1586752557190.mp4':
Metadata:
encoder : Lavf58.29.100
Stream #0:0: Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 608x1080, q=2-31, 1378 kb/s, 30 fps, 15360 tbn, 30 tbc
Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 64 kb/s
12:36:01.277 [main] WARN com.b612kaji.h5.api.TestMain - !!! end cvQueryFrame..video:https://abc.com/media_1553508014839.mp4
[mp4 @ 0x7fcf6db63200] Starting second pass: moving the moov atom to the beginning of the file
[libx264 @ 0x7fcf6c31e600] frame I:2 Avg QP:24.66 size: 20525
[libx264 @ 0x7fcf6c31e600] frame P:87 Avg QP:26.86 size: 10054
[libx264 @ 0x7fcf6c31e600] frame B:189 Avg QP:28.66 size: 3569
[libx264 @ 0x7fcf6c31e600] consecutive B-frames: 6.1% 7.2% 7.6% 79.1%
[libx264 @ 0x7fcf6c31e600] mb I I16..4: 24.5% 61.5% 14.0%
[libx264 @ 0x7fcf6c31e600] mb P I16..4: 16.8% 24.9% 2.1% P16..4: 24.9% 8.4% 1.9% 0.0% 0.0% skip:21.1%
[libx264 @ 0x7fcf6c31e600] mb B I16..4: 3.6% 3.6% 0.1% B16..8: 30.9% 4.6% 0.1% direct:10.9% skip:46.1% L0:45.7% L1:49.7% BI: 4.6%
[libx264 @ 0x7fcf6c31e600] final ratefactor: 25.73
[libx264 @ 0x7fcf6c31e600] 8x8 transform intra:55.1% inter:55.9%
[libx264 @ 0x7fcf6c31e600] coded y,uvDC,uvAC intra: 35.6% 21.2% 3.6% inter: 10.2% 5.1% 0.1%
[libx264 @ 0x7fcf6c31e600] i16 v,h,dc,p: 40% 21% 20% 18%
[libx264 @ 0x7fcf6c31e600] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 27% 13% 28% 5% 5% 7% 5% 7% 4%
[libx264 @ 0x7fcf6c31e600] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 28% 12% 16% 7% 7% 9% 5% 12% 5%
[libx264 @ 0x7fcf6c31e600] i8c dc,h,v,p: 73% 11% 14% 2%
[libx264 @ 0x7fcf6c31e600] Weighted P-Frames: Y:8.0% UV:1.1%
[libx264 @ 0x7fcf6c31e600] ref P L0: 78.2% 21.8%
[libx264 @ 0x7fcf6c31e600] ref B L0: 88.4% 11.6%
[libx264 @ 0x7fcf6c31e600] ref B L1: 97.2% 2.8%
[libx264 @ 0x7fcf6c31e600] kb/s:1372.88
[aac @ 0x7fcf6db64400] Qavg: nan
12:36:01.563 [main] INFO com.b612kaji.h5.api.TestMain - 耗时：6834 ms
Process finished with exit code 0
after I used ffmpegframefilter, the generated video had no sound.
GeneralLHW
on 13 Apr 2020
You'll need to setAudioChannels() to something greater than 0 to have audio.
saudet
on 13 Apr 2020
You'll need to
setAudioChannels()to something greater than 0 to have audio.
i add that:
fFmpegFrameFilter.setAudioChannels(frameGrabber.getAudioChannels());
the log:
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'https://abc.com/media_1553508014839.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
title : B612咔叽
encoder : Lavf57.71.100
genre : {"version":"8.1.1","recordTime":"11","sections":"(1, 1)","frames":330,"sectionId":28}
Duration: 00:00:11.21, start: 0.000000, bitrate: 5382 kb/s
Stream #0:0(eng): Video: h264 (Baseline) (avc1 / 0x31637661), yuv420p(tv, smpte170m/bt470bg/smpte170m), 720x1280, 5247 kb/s, SAR 1:1 DAR 9:16, 29.89 fps, 90k tbr, 90k tbn, 180k tbc (default)
Metadata:
handler_name : VideoHandler
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default)
Metadata:
handler_name : SoundHandler
13:56:38.163 [main] INFO com.b612kaji.h5.api.TestMain - wight:720,height:1280,fileName:/Users/mac/Desktop/B612/2bc/1586757398161.mp4
[libx264 @ 0x7f8845334800] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x7f8845334800] profile High, level 3.1, 4:2:0, 8-bit
[libx264 @ 0x7f8845334800] 264 - core 157 - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=1 ref=2 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=4 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=12 lookahead_threads=3 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=1 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=20 rc=abr mbtree=1 bitrate=1378 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to '/Users/mac/Desktop/B612/2bc/1586757398161.mp4':
Metadata:
encoder : Lavf58.29.100
Stream #0:0: Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 608x1080, q=2-31, 1378 kb/s, 30 fps, 15360 tbn, 30 tbc
Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 64 kb/s
13:56:42.867 [main] WARN com.b612kaji.h5.api.TestMain - !!! end cvQueryFrame..video:https://abc.com/media_1553508014839.mp4
[mp4 @ 0x7f8844a10e00] Starting second pass: moving the moov atom to the beginning of the file
[libx264 @ 0x7f8845334800] frame I:2 Avg QP:24.66 size: 20525
[libx264 @ 0x7f8845334800] frame P:87 Avg QP:26.86 size: 10054
[libx264 @ 0x7f8845334800] frame B:189 Avg QP:28.66 size: 3569
[libx264 @ 0x7f8845334800] consecutive B-frames: 6.1% 7.2% 7.6% 79.1%
[libx264 @ 0x7f8845334800] mb I I16..4: 24.5% 61.5% 14.0%
[libx264 @ 0x7f8845334800] mb P I16..4: 16.8% 24.9% 2.1% P16..4: 24.9% 8.4% 1.9% 0.0% 0.0% skip:21.1%
[libx264 @ 0x7f8845334800] mb B I16..4: 3.6% 3.6% 0.1% B16..8: 30.9% 4.6% 0.1% direct:10.9% skip:46.1% L0:45.7% L1:49.7% BI: 4.6%
[libx264 @ 0x7f8845334800] final ratefactor: 25.73
[libx264 @ 0x7f8845334800] 8x8 transform intra:55.1% inter:55.9%
[libx264 @ 0x7f8845334800] coded y,uvDC,uvAC intra: 35.6% 21.2% 3.6% inter: 10.2% 5.1% 0.1%
[libx264 @ 0x7f8845334800] i16 v,h,dc,p: 40% 21% 20% 18%
[libx264 @ 0x7f8845334800] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 27% 13% 28% 5% 5% 7% 5% 7% 4%
[libx264 @ 0x7f8845334800] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 28% 12% 16% 7% 7% 9% 5% 12% 5%
[libx264 @ 0x7f8845334800] i8c dc,h,v,p: 73% 11% 14% 2%
[libx264 @ 0x7f8845334800] Weighted P-Frames: Y:8.0% UV:1.1%
[libx264 @ 0x7f8845334800] ref P L0: 78.2% 21.8%
[libx264 @ 0x7f8845334800] ref B L0: 88.4% 11.6%
[libx264 @ 0x7f8845334800] ref B L1: 97.2% 2.8%
[libx264 @ 0x7f8845334800] kb/s:1372.88
[aac @ 0x7f8845336000] Qavg: nan
still no sound.
cry...
GeneralLHW
on 13 Apr 2020
You're not passing any audio frames to FFmpegFrameFilter, and you'll probably also need to pass a graph for the audio.
saudet
on 13 Apr 2020
You're not passing any audio frames to FFmpegFrameFilter, and you'll probably also need to pass a graph for the audio.
But how should the code be written?That's how I write, but there's still no audio
Frame capturedFrame;
while (true) {
try {
capturedFrame = frameGrabber.grabFrame();
if (capturedFrame == null) {
log.warn("!!! end cvQueryFrame..video:{}",path);
break;
}
if (capturedFrame.image != null) {
fFmpegFrameFilter.push(capturedFrame);
if (fFmpegFrameFilter.pull() != null) {
recorder.record(capturedFrame);
}
}
// This is my new add code
if (capturedFrame.samples != null){
fFmpegFrameFilter.push(capturedFrame);// push samples??
if (fFmpegFrameFilter.pull() != null) {// pull samples?
recorder.record(capturedFrame);
}
}
} catch (Exception e) {
e.printStackTrace();
}
}
I'm not sure how the API calls.Give me some advice, thank you
GeneralLHW
on 13 Apr 2020
There's an example of this here: https://github.com/bytedeco/javacv/blob/master/platform/src/test/java/org/bytedeco/javacv/FrameFilterTest.java
saudet
on 13 Apr 2020
ok,Can you help me see that
if (capturedFrame.image != null || capturedFrame.samples != null) {
fFmpegFrameFilter.push(capturedFrame);
}
if ( (pullFrame = fFmpegFrameFilter.pull()) != null) {
if(pullFrame.image != null || pullFrame.samples != null){
recorder.record(pullFrame);
}
}
I think so, but the result is still the same
GeneralLHW
on 13 Apr 2020
You'll need to show me your your code if you want me to check what's wrong with it.
saudet
on 13 Apr 2020
You'll need to show me your your code if you want me to check what's wrong with it.
This is my code.Please Check it for me. Thank you very much
public static void videoTrans(String path) throws Exception{
FFmpegFrameGrabber frameGrabber = new FFmpegFrameGrabber(path);
String fileName;
FFmpegFrameRecorder recorder;
try {
frameGrabber.start();
fileName = "/Users/mac/Desktop/B612/2bc/"+System.currentTimeMillis()+".mp4";
FFmpegFrameFilter fFmpegFrameFilter = new FFmpegFrameFilter("fps=fps=25",frameGrabber.getImageWidth(), frameGrabber.getImageHeight());
fFmpegFrameFilter.setAudioChannels(frameGrabber.getAudioChannels());
// fFmpegFrameFilter.setPixelFormat(avutil.AV_PIX_FMT_YUV420P);
fFmpegFrameFilter.setSampleFormat(frameGrabber.getSampleFormat());
fFmpegFrameFilter.setSampleRate(frameGrabber.getSampleRate());
fFmpegFrameFilter.setPixelFormat(frameGrabber.getPixelFormat());
fFmpegFrameFilter.start();
int height = frameGrabber.getImageHeight();
int width = frameGrabber.getImageWidth();
if (height > 1080) {
width = (1080 * width) / height;
if (width % 2 != 0) {
width++;
}
height = 1080;
}
int videoBitRate = (int)((width*height*30)*1*0.07);
recorder = new FFmpegFrameRecorder(fileName, width, height, frameGrabber.getAudioChannels());
recorder.setSampleRate(frameGrabber.getSampleRate());
recorder.setAudioChannels(frameGrabber.getAudioChannels());
recorder.setVideoOption("preset", "faster");
recorder.setVideoOption("strict", "experimental");
recorder.setOption("movflags", "faststart");
// yuv420p,像素
recorder.setPixelFormat(avutil.AV_PIX_FMT_YUV420P);
recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);
recorder.setAudioCodec(avcodec.AV_CODEC_ID_AAC);
recorder.setFormat("mp4");
//比特
recorder.setVideoBitrate(videoBitRate);
recorder.start();
Frame capturedFrame;
Frame pullFrame;
while (true) {
try {
capturedFrame = frameGrabber.grab();
if (capturedFrame == null) {
log.warn("!!! end cvQueryFrame..video:{}",path);
break;
}
if (capturedFrame.image != null || capturedFrame.samples != null) {
fFmpegFrameFilter.push(capturedFrame);
}
if ( (pullFrame = fFmpegFrameFilter.pull()) != null) {
if(pullFrame.image != null || pullFrame.samples != null){
recorder.record(pullFrame);
}
}
} catch (Exception e) {
e.printStackTrace();
}
}
recorder.stop();
recorder.release();
frameGrabber.stop();
fFmpegFrameFilter.stop();
} catch (Exception e) {
e.printStackTrace();
}
}
log:
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'https://abc.com/media_1553508014839.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
title : B612咔叽
encoder : Lavf57.71.100
genre : {"version":"8.1.1","recordTime":"11","sections":"(1, 1)","frames":330,"sectionId":28}
Duration: 00:00:11.21, start: 0.000000, bitrate: 5382 kb/s
Stream #0:0(eng): Video: h264 (Baseline) (avc1 / 0x31637661), yuv420p(tv, smpte170m/bt470bg/smpte170m), 720x1280, 5247 kb/s, SAR 1:1 DAR 9:16, 29.89 fps, 90k tbr, 90k tbn, 180k tbc (default)
Metadata:
handler_name : VideoHandler
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default)
Metadata:
handler_name : SoundHandler
[libx264 @ 0x7f8e20bddc00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x7f8e20bddc00] profile High, level 3.1, 4:2:0, 8-bit
[libx264 @ 0x7f8e20bddc00] 264 - core 157 - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=1 ref=2 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=4 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=12 lookahead_threads=3 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=1 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=20 rc=abr mbtree=1 bitrate=1378 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to '/Users/mac/Desktop/B612/2bc/1586774878647.mp4':
Metadata:
encoder : Lavf58.29.100
Stream #0:0: Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 608x1080, q=2-31, 1378 kb/s, 30 fps, 15360 tbn, 30 tbc
Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 64 kb/s
18:48:02.294 [main] WARN com.b612kaji.h5.api.TestMain - !!! end cvQueryFrame..video:https://abc.com/media_1553508014839.mp4
[mp4 @ 0x7f8e209b2000] Starting second pass: moving the moov atom to the beginning of the file
[libx264 @ 0x7f8e20bddc00] frame I:2 Avg QP:25.03 size: 20228
[libx264 @ 0x7f8e20bddc00] frame P:84 Avg QP:26.83 size: 10315
[libx264 @ 0x7f8e20bddc00] frame B:192 Avg QP:28.64 size: 3566
[libx264 @ 0x7f8e20bddc00] consecutive B-frames: 4.7% 5.8% 11.9% 77.7%
[libx264 @ 0x7f8e20bddc00] mb I I16..4: 26.1% 59.7% 14.2%
[libx264 @ 0x7f8e20bddc00] mb P I16..4: 17.1% 25.5% 2.2% P16..4: 25.3% 8.5% 1.9% 0.0% 0.0% skip:19.4%
[libx264 @ 0x7f8e20bddc00] mb B I16..4: 3.7% 3.6% 0.1% B16..8: 30.9% 4.6% 0.1% direct:11.0% skip:46.1% L0:45.8% L1:49.8% BI: 4.4%
[libx264 @ 0x7f8e20bddc00] final ratefactor: 25.71
[libx264 @ 0x7f8e20bddc00] 8x8 transform intra:54.9% inter:55.7%
[libx264 @ 0x7f8e20bddc00] coded y,uvDC,uvAC intra: 35.6% 21.1% 3.5% inter: 10.2% 5.1% 0.1%
[libx264 @ 0x7f8e20bddc00] i16 v,h,dc,p: 41% 21% 20% 18%
[libx264 @ 0x7f8e20bddc00] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 28% 12% 27% 5% 5% 7% 5% 7% 4%
[libx264 @ 0x7f8e20bddc00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 28% 11% 16% 7% 7% 9% 5% 12% 5%
[libx264 @ 0x7f8e20bddc00] i8c dc,h,v,p: 73% 11% 13% 2%
[libx264 @ 0x7f8e20bddc00] Weighted P-Frames: Y:7.1% UV:0.0%
[libx264 @ 0x7f8e20bddc00] ref P L0: 76.3% 23.7%
[libx264 @ 0x7f8e20bddc00] ref B L0: 88.1% 11.9%
[libx264 @ 0x7f8e20bddc00] ref B L1: 97.0% 3.0%
[libx264 @ 0x7f8e20bddc00] kb/s:1374.01
[aac @ 0x7f8e22adde00] Qavg: nan
Try something like this:
FFmpegFrameFilter fFmpegFrameFilter = new FFmpegFrameFilter("fps=fps=25", "anull", frameGrabber.getImageWidth(), frameGrabber.getImageHeight(), frameGrabber.getAudioChannels());
FFmpegFrameFilter fFmpegFrameFilter = new FFmpegFrameFilter("fps=fps=25", "anull", frameGrabber.getImageWidth(), frameGrabber.getImageHeight(), frameGrabber.getAudioChannels());
Thank you. I tried. The sound is normal now, but the image is playing fast. It's very obvious that the sound and picture are not synchronized. And I ffprobe for a moment, and found that the FPS of the generated video is not the 25 I set, but the 30 FPS
GeneralLHW
on 14 Apr 2020
You'll also need to call FFmpegFrameRecorder.setFrameRate(25).
saudet
on 14 Apr 2020
setFrameRate
Thank you. After I added it, FPS changed to 25. This video seems to have no problem. But after I changed another video, I found that the image played fast, and the sound and picture were still very obvious out of sync
log：
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'https://abc.com/media_1579249700648.mp4':
Metadata:
major_brand : mp42
minor_version : 0
compatible_brands: mp42mp41
creation_time : 2019-12-23T08:05:25.000000Z
Duration: 00:00:32.40, start: 0.000000, bitrate: 3885 kb/s
Stream #0:0(eng): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709), 1080x1920, 3533 kb/s, 25 fps, 25 tbr, 25k tbn, 50 tbc (default)
Metadata:
creation_time : 2019-12-23T08:05:25.000000Z
handler_name : ?Mainconcept Video Media Handler
encoder : AVC Coding
Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 317 kb/s (default)
Metadata:
creation_time : 2019-12-23T08:05:26.000000Z
handler_name : #Mainconcept MP4 Sound Media Handler
[libx264 @ 0x7fa96eb4d200] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x7fa96eb4d200] profile High, level 3.1, 4:2:0, 8-bit
[libx264 @ 0x7fa96eb4d200] 264 - core 157 - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=1 ref=2 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=4 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=12 lookahead_threads=3 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=1 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=20 rc=abr mbtree=1 bitrate=1378 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to '/Users/mac/Desktop/B612/2bc/1586832064009.mp4':
Metadata:
encoder : Lavf58.29.100
Stream #0:0: Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 608x1080, q=2-31, 1378 kb/s, 25 fps, 12800 tbn, 25 tbc
Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 64 kb/s
10:41:14.156 [main] WARN com.b612kaji.h5.api.TestMain - !!! end cvQueryFrame..video:https://abc.com/media_1579249700648.mp4
[mp4 @ 0x7fa96eb4c000] Starting second pass: moving the moov atom to the beginning of the file
[libx264 @ 0x7fa96eb4d200] frame I:35 Avg QP:14.55 size: 33121
[libx264 @ 0x7fa96eb4d200] frame P:221 Avg QP:18.42 size: 10452
[libx264 @ 0x7fa96eb4d200] frame B:418 Avg QP:20.52 size: 2037
[libx264 @ 0x7fa96eb4d200] consecutive B-frames: 13.5% 8.9% 7.6% 70.0%
[libx264 @ 0x7fa96eb4d200] mb I I16..4: 44.2% 32.0% 23.8%
[libx264 @ 0x7fa96eb4d200] mb P I16..4: 15.8% 12.0% 2.8% P16..4: 18.2% 8.0% 3.1% 0.0% 0.0% skip:40.1%
[libx264 @ 0x7fa96eb4d200] mb B I16..4: 1.9% 1.4% 0.1% B16..8: 14.7% 4.0% 0.3% direct: 5.0% skip:72.6% L0:43.8% L1:46.9% BI: 9.3%
[libx264 @ 0x7fa96eb4d200] final ratefactor: 19.50
[libx264 @ 0x7fa96eb4d200] 8x8 transform intra:37.2% inter:32.5%
[libx264 @ 0x7fa96eb4d200] coded y,uvDC,uvAC intra: 37.2% 28.0% 8.2% inter: 6.9% 6.0% 0.1%
[libx264 @ 0x7fa96eb4d200] i16 v,h,dc,p: 61% 26% 9% 5%
[libx264 @ 0x7fa96eb4d200] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 29% 26% 19% 4% 4% 4% 4% 5% 4%
[libx264 @ 0x7fa96eb4d200] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 31% 23% 12% 7% 5% 6% 5% 7% 5%
[libx264 @ 0x7fa96eb4d200] i8c dc,h,v,p: 67% 15% 15% 2%
[libx264 @ 0x7fa96eb4d200] Weighted P-Frames: Y:16.7% UV:15.8%
[libx264 @ 0x7fa96eb4d200] ref P L0: 78.7% 21.3%
[libx264 @ 0x7fa96eb4d200] ref B L0: 84.1% 15.9%
[libx264 @ 0x7fa96eb4d200] ref B L1: 94.7% 5.3%
[libx264 @ 0x7fa96eb4d200] kb/s:1282.13
[aac @ 0x7fa96eb42600] Qavg: 469.241
You can see my original video and converted video from the following link address：
url :https://pan.baidu.com/s/1gZFmcQ7XBXOISNh8Cm7lBA password:r8lh
GeneralLHW
on 14 Apr 2020
And FFmpegFrameFilter.setFrameRate(FFmpegFrameGrabber.getFrameRate()) too.
saudet
on 14 Apr 2020
And
FFmpegFrameFilter.setFrameRate(FFmpegFrameGrabber.getFrameRate())too.
Yes, after I added it, I found that the video is normal. The code is as follows:
public static void videoTrans(String path) throws Exception{
FFmpegFrameGrabber frameGrabber = new FFmpegFrameGrabber(path);
String fileName;
FFmpegFrameRecorder recorder;
try {
frameGrabber.start();
fileName = "/Users/mac/Desktop/B612/2bc/"+System.currentTimeMillis()+".mp4";
// 1.set fps
FFmpegFrameFilter fFmpegFrameFilter = new FFmpegFrameFilter("fps=fps=25", "anull", frameGrabber.getImageWidth(), frameGrabber.getImageHeight(), frameGrabber.getAudioChannels());
fFmpegFrameFilter.setSampleFormat(frameGrabber.getSampleFormat());
fFmpegFrameFilter.setSampleRate(frameGrabber.getSampleRate());
fFmpegFrameFilter.setPixelFormat(frameGrabber.getPixelFormat());
// 2.set fps
fFmpegFrameFilter.setFrameRate(frameGrabber.getFrameRate());
fFmpegFrameFilter.start();
int height = frameGrabber.getImageHeight();
int width = frameGrabber.getImageWidth();
if (height > 1080) {
width = (1080 * width) / height;
if (width % 2 != 0) {
width++;
}
height = 1080;
}
int videoBitRate = (int)((width*height*30)*1*0.07);
recorder = new FFmpegFrameRecorder(fileName, width, height, frameGrabber.getAudioChannels());
recorder.setSampleRate(frameGrabber.getSampleRate());
recorder.setAudioChannels(frameGrabber.getAudioChannels());
recorder.setVideoOption("preset", "faster");
recorder.setVideoOption("strict", "experimental");
recorder.setOption("movflags", "faststart");
// 3.set fps
recorder.setFrameRate(25);
// yuv420p,像素
recorder.setPixelFormat(avutil.AV_PIX_FMT_YUV420P);
recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);
recorder.setAudioCodec(avcodec.AV_CODEC_ID_AAC);
recorder.setFormat("mp4");
//比特
recorder.setVideoBitrate(videoBitRate);
recorder.start();
Frame capturedFrame;
Frame pullFrame;
while (true) {
try {
capturedFrame = frameGrabber.grab();
if (capturedFrame == null) {
log.warn("!!! end cvQueryFrame..video:{}",path);
break;
}
if (capturedFrame.image != null || capturedFrame.samples != null) {
fFmpegFrameFilter.push(capturedFrame);
}
if ( (pullFrame = fFmpegFrameFilter.pull()) != null) {
if(pullFrame.image != null || pullFrame.samples != null){
recorder.record(pullFrame);
}
}
} catch (Exception e) {
e.printStackTrace();
}
}
recorder.stop();
recorder.release();
frameGrabber.stop();
fFmpegFrameFilter.stop();
} catch (Exception e) {
e.printStackTrace();
}
}
Is frame rate setting indispensable in three places of my code annotation?
GeneralLHW
on 14 Apr 2020
Yes, FFmpegFrameGrabber, FFmpegFrameFilter, and FFmpegFrameRecorder function independently from each other.
saudet
on 14 Apr 2020
Yes,
FFmpegFrameGrabber,FFmpegFrameFilter, andFFmpegFrameRecorderfunction independently from each other.
Yes, thank you very much,Thanks a lot.
GeneralLHW
on 14 Apr 2020
Related issues
chenhl05
·
4Comments
y4nnick
·
3Comments
cansik
·
4Comments
nghiepvth
·
3Comments
chrisliu12345
·
4Comments