Javacv: Right way to implement overlay filter
I'm currently working on adding an overlay image on top of the video.
It is clear, how to do it on bare FFMpeg, but what is the right way to provide an image for the FFMpegFrameFilter for an overlay filter?
Is it possible to write there -i path/to/img.jpg -filter_complex "[0][1]overlay"? Or there is another way?
And what is the shortcut for a video stream (0 or 1 in this case)?
I know, that it sounds more like a StackOverflow question but it looks like it is impossible to get a reply on a JavaCV topic there
bug
DeKinci
All 111 comments
So I used movie filter and now string looks like this: movie=path/to/img.jpg[img];[0][img]overlay and all i got is just rotated vertically and tripled image
DeKinci
on 21 Mar 2018
Are you sure you are getting the output in the right pixel format? See
issue #887.
saudet
on 21 Mar 2018
Yes, the image is transformed exactly like that. I will adjust it and share the result.
But another problem is that the picture is not presented in the final video in any way (not transformed, not tripled, just no image).
Now I use [in] shortcut to specify the input stream, [0] refused to work. I do not specify [out] explicitly, because it is not necessary in the common ffmpeg and it also causes crash.
The path that I provide to the overlay filter is a full path /storage/emulated/0/Appname/img.png
When I only use movie=path filter it also crashes
By the way, maybe it would be useful to extract some frequently used filters into classes and also provide API for constructing a filters graph?
DeKinci
on 22 Mar 2018
Yes, anything to make it easier to use or documentation is welcome! Thanks
saudet
on 22 Mar 2018
BTW, this sounds like a duplicate of issue #667 and I'm pretty sure that's fixed by adjusting the pixel formats accordingly. If this isn't the case though, please let me know.
saudet
on 22 Mar 2018
Firstly, I tested ffmpeg on a pc with the following command and it worked fine: ffmpeg -i video.mp4 -vf movie=img.png[img];[in][img]overlay[out] output.mp4 so it is possible to assert that filter string is OK.
Next, setting pixelFormat in Recorder.record didn't actually help, but settings right pixelFormat to Filter solved the problem of tripled and turned video (it is not necessary to specify format in recorder).
Now filter's setup looks like this:
filter = new FFmpegFrameFilter("movie=" + path + "[img];[in][img]overlay[out]", imageWidth, imageHeight);
filter.setPixelFormat(avutil.AV_PIX_FMT_YUV420P);
filter.start();
The path is specified correctly and looks like /storage/emulated/0/AppName/test.jpg. I tried both jpg and png formats. When I specify the wrong path e.g. /storage/emulated/0/nothing/test.jpg it crashes with the following error:
org.bytedeco.javacv.FrameFilter$Exception: avfilter_graph_create_filter(): Cannot create buffer source.
at org.bytedeco.javacv.FFmpegFrameFilter.startUnsafe(FFmpegFrameFilter.java:191)
So the path is also OK.
BUT there is still no image on the final video!
The way I enque and get frames is like this:
((ByteBuffer) frame.image[0].position(0)).put(bytes);
filterManager.put(frame);
Frame edited = filterManager.get();
if (edited != null) {
//Recording stuff here
}
where filterManager just pushs and pulls frames in try-catch to the filter.
I tried to add an offset to the image in case if it is outside of the screen: [in][img]overlay=x=200:y=200[out] but this also didn't helped
DeKinci
on 22 Mar 2018
Maybe we could get more information about what is happening by increasing the log verbosity?
saudet
on 22 Mar 2018
Anyway, if you could call av_log_set_level(AV_LOG_DEBUG) and post the output of that after processing for 1 or 2 frames, it should help debug this. Thanks!
saudet
on 23 Mar 2018
av_log_set_level(AV_LOG_DEBUG) didn't worked, as well as setting another levels. There is no output related to anything except the app. I call it before I call any javaCV's method.
Also, I tried to switch overlay layers from [in][img] to [img][in] and got black frames result and also tried to input mp4 video as movie instead of a pic and result is the same as with the picture
DeKinci
on 24 Mar 2018
Also make sure to call FFmpegLogCallback.set() if you don't see anything on the console like that.
saudet
on 24 Mar 2018
Thanks! I got this for overlaying video (it looks same as image):
03-25 03:08:21.848 3112-3156/com.domain.appname I/System.out: detected 4 logical cores
03-25 03:08:21.849 3112-3156/com.domain.appname I/System.out: [in @ 0xe86adb40] Setting 'video_size' to value '1280x720'
03-25 03:08:21.849 3112-3156/com.domain.appname I/System.out: [in @ 0xe86adb40] Setting 'pix_fmt' to value '0'
03-25 03:08:21.849 3112-3156/com.domain.appname I/System.out: [in @ 0xe86adb40] Setting 'time_base' to value '1/30'
03-25 03:08:21.849 3112-3156/com.domain.appname I/System.out: [in @ 0xe86adb40] Setting 'pixel_aspect' to value '1/1'
03-25 03:08:21.850 3112-3156/com.domain.appname I/System.out: [in @ 0xe86adb40] w:1280 h:720 pixfmt:yuv420p tb:1/30 fr:0/1 sar:1/1 sws_param:
03-25 03:08:21.850 3112-3156/com.domain.appname I/System.out: [Parsed_movie_0 @ 0xe86add20] Setting 'filename' to value '/storage/emulated/0/AppName/test.mp4'
03-25 03:08:21.851 3112-3156/com.domain.appname I/System.out: [NULL @ 0xd924b200] Opening '/storage/emulated/0/AppName/test.mp4' for reading
03-25 03:08:21.851 3112-3156/com.domain.appname I/System.out: [file @ 0xd9adbe70] Setting default whitelist 'file,crypto'
03-25 03:08:21.851 3112-3156/com.domain.appname I/System.out: [mov,mp4,m4a,3gp,3g2,mj2 @ 0xd924b200] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
03-25 03:08:21.851 3112-3156/com.domain.appname I/System.out: [mov,mp4,m4a,3gp,3g2,mj2 @ 0xd924b200] ISO: File Type Major Brand: isom
03-25 03:08:21.852 3112-3156/com.domain.appname I/System.out: [mov,mp4,m4a,3gp,3g2,mj2 @ 0xd924b200] Unknown dref type 0x206c7275 size 12
03-25 03:08:21.852 3112-3156/com.domain.appname I/System.out: [mov,mp4,m4a,3gp,3g2,mj2 @ 0xd924b200] Processing st: 0, edit list 0 - media time: -1, duration: 1014
03-25 03:08:21.852 3112-3156/com.domain.appname I/System.out: [mov,mp4,m4a,3gp,3g2,mj2 @ 0xd924b200] Processing st: 0, edit list 1 - media time: 0, duration: 371712
03-25 03:08:21.852 3112-3156/com.domain.appname I/System.out: [mov,mp4,m4a,3gp,3g2,mj2 @ 0xd924b200] drop a frame at curr_cts: 371712 @ 422
03-25 03:08:21.853 3112-3156/com.domain.appname I/System.out: [mov,mp4,m4a,3gp,3g2,mj2 @ 0xd924b200] Unknown dref type 0x206c7275 size 12
03-25 03:08:21.853 3112-3156/com.domain.appname I/System.out: [mov,mp4,m4a,3gp,3g2,mj2 @ 0xd924b200] Processing st: 1, edit list 0 - media time: 1024, duration: 1073129
03-25 03:08:21.853 3112-3156/com.domain.appname I/System.out: [mov,mp4,m4a,3gp,3g2,mj2 @ 0xd924b200] drop a frame at curr_cts: 0 @ 0
03-25 03:08:21.853 3112-3156/com.domain.appname I/System.out: [mov,mp4,m4a,3gp,3g2,mj2 @ 0xd924b200] Before avformat_find_stream_info() pos: 1693560 bytes read:48607 seeks:1 nb_streams:2
03-25 03:08:21.862 3112-3156/com.domain.appname I/System.out: [mov,mp4,m4a,3gp,3g2,mj2 @ 0xd924b200] demuxer injecting skip 1024 / discard 0
03-25 03:08:21.862 3112-3156/com.domain.appname I/System.out: [aac @ 0xe86cc400] skip 1024 / discard 0 samples due to side data
03-25 03:08:21.863 3112-3156/com.domain.appname I/System.out: [mov,mp4,m4a,3gp,3g2,mj2 @ 0xd924b200] All info found
03-25 03:08:21.864 3112-3156/com.domain.appname I/System.out: [mov,mp4,m4a,3gp,3g2,mj2 @ 0xd924b200] After avformat_find_stream_info() pos: 30647 bytes read:81375 seeks:2 frames:5
03-25 03:08:21.864 3112-3156/com.domain.appname I/System.out: [Parsed_movie_0 @ 0xe86add20] seek_point:0 format_name:(null) file_name:/storage/emulated/0/AppName/test.mp4 stream_index:-1
03-25 03:08:21.865 3112-3156/com.domain.appname I/System.out: [auto_scaler_0 @ 0xe86adcc0] w:iw h:ih flags:'bilinear' interl:0
03-25 03:08:21.865 3112-3156/com.domain.appname I/System.out: [Parsed_overlay_1 @ 0xe86ade40] auto-inserting filter 'auto_scaler_0' between the filter 'in' and the filter 'Parsed_overlay_1'
03-25 03:08:21.865 3112-3156/com.domain.appname I/System.out: [AVFilterGraph @ 0xd90b5900] query_formats: 4 queried, 2 merged, 1 already done, 0 delayed
03-25 03:08:21.865 3112-3156/com.domain.appname I/System.out: [auto_scaler_0 @ 0xe86adcc0] w:1280 h:720 fmt:yuv420p sar:1/1 -> w:1280 h:720 fmt:yuva420p sar:1/1 flags:0x2
03-25 03:08:21.865 3112-3156/com.domain.appname I/System.out: [Parsed_overlay_1 @ 0xe86ade40] main w:1280 h:720 fmt:yuv420p overlay w:1280 h:720 fmt:yuva420p
03-25 03:08:21.865 3112-3156/com.domain.appname I/System.out: [Parsed_overlay_1 @ 0xe86ade40] [framesync @ 0xe11436a0] Selected 1/15360 time base
03-25 03:08:21.865 3112-3156/com.domain.appname I/System.out: [Parsed_overlay_1 @ 0xe86ade40] [framesync @ 0xe11436a0] Sync level 2
03-25 03:08:21.939 3112-3156/com.domain.appname I/System.out: [libx264 @ 0xe86ccc00] using cpu capabilities: none!
03-25 03:08:21.939 3112-3156/com.domain.appname I/System.out: [libx264 @ 0xe86ccc00] using cpu capabilities: none!
03-25 03:08:21.946 3112-3156/com.domain.appname I/System.out: [libx264 @ 0xe86ccc00] profile Constrained Baseline, level 3.1
03-25 03:08:21.946 3112-3156/com.domain.appname I/System.out: [libx264 @ 0xe86ccc00] profile Constrained Baseline, level 3.1
03-25 03:08:21.946 3112-3156/com.domain.appname I/System.out: [libx264 @ 0xe86ccc00] 264 - core 152 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=4 lookahead_threads=4 sliced_threads=1 slices=4 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=60 keyint_min=6 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=27.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
03-25 03:08:21.946 3112-3156/com.domain.appname I/System.out: [libx264 @ 0xe86ccc00] 264 - core 152 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=4 lookahead_threads=4 sliced_threads=1 slices=4 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=60 keyint_min=6 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=27.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
03-25 03:08:21.999 3112-3156/com.domain.appname I/System.out: [rtmp @ 0xd9adc260] No default whitelist set
03-25 03:08:21.999 3112-3156/com.domain.appname I/System.out: [tcp @ 0xe86ae2c0] No default whitelist set
03-25 03:08:25.022 3112-3156/com.domain.appname I/System.out: [rtmp @ 0xd9adc260] Handshaking...
03-25 03:08:25.050 3112-3156/com.domain.appname I/System.out: [rtmp @ 0xd9adc260] Type answer 3
03-25 03:08:25.050 3112-3156/com.domain.appname I/System.out: [rtmp @ 0xd9adc260] Server version 4.0.0.1
03-25 03:08:25.050 3112-3156/com.domain.appname I/System.out: [rtmp @ 0xd9adc260] Proto = rtmp, path = /live2/qc8s-14g7-q31u-am57, app = live2, fname = qc8s-14g7-q31u-am57
03-25 03:08:25.281 3112-3156/com.domain.appname I/System.out: [rtmp @ 0xd9adc260] Window acknowledgement size = 2500000
03-25 03:08:25.281 3112-3156/com.domain.appname I/System.out: [rtmp @ 0xd9adc260] Max sent, unacked = 10000000
03-25 03:08:25.281 3112-3156/com.domain.appname I/System.out: [rtmp @ 0xd9adc260] New incoming chunk size = 256
03-25 03:08:25.314 3112-3156/com.domain.appname I/System.out: [rtmp @ 0xd9adc260] Releasing stream...
03-25 03:08:25.314 3112-3156/com.domain.appname I/System.out: [rtmp @ 0xd9adc260] FCPublish stream...
03-25 03:08:25.314 3112-3156/com.domain.appname I/System.out: [rtmp @ 0xd9adc260] Creating stream...
03-25 03:08:25.342 3112-3156/com.domain.appname I/System.out: [rtmp @ 0xd9adc260] Sending publish command for 'qc8s-14g7-q31u-am57'
03-25 03:08:25.577 3112-3156/com.domain.appname I/System.out: Output #0, flv, to 'rtmp://a.rtmp.youtube.com/live2/qc8s-14g7-q31u-am57':
03-25 03:08:25.577 3112-3156/com.domain.appname I/System.out: Output #0, flv, to 'rtmp://a.rtmp.youtube.com/live2/qc8s-14g7-q31u-am57':
03-25 03:08:25.577 3112-3156/com.domain.appname I/System.out: Metadata:
03-25 03:08:25.577 3112-3156/com.domain.appname I/System.out: Metadata:
03-25 03:08:25.577 3112-3156/com.domain.appname I/System.out: encoder :
03-25 03:08:25.577 3112-3156/com.domain.appname I/System.out: encoder :
03-25 03:08:25.577 3112-3156/com.domain.appname I/System.out: Lavf57.83.100
03-25 03:08:25.577 3112-3156/com.domain.appname I/System.out: Lavf57.83.100
03-25 03:08:25.577 3112-3156/com.domain.appname I/System.out: Stream #0:0
03-25 03:08:25.577 3112-3156/com.domain.appname I/System.out: Stream #0:0
03-25 03:08:25.577 3112-3156/com.domain.appname I/System.out: , 0, 1/1000
03-25 03:08:25.577 3112-3156/com.domain.appname I/System.out: : Video: h264 (Constrained Baseline), 1 reference frame ([7][0][0][0] / 0x0007), yuv420p, 1280x720 (0x0), 0/1, q=2-31, 2764 kb/s
03-25 03:08:25.577 3112-3156/com.domain.appname I/System.out: : Video: h264 (Constrained Baseline), 1 reference frame ([7][0][0][0] / 0x0007), yuv420p, 1280x720 (0x0), 0/1, q=2-31, 2764 kb/s
03-25 03:08:25.577 3112-3156/com.domain.appname I/System.out: ,
03-25 03:08:25.577 3112-3156/com.domain.appname I/System.out: ,
03-25 03:08:25.578 3112-3156/com.domain.appname I/System.out: 1k tbn
03-25 03:08:25.578 3112-3156/com.domain.appname I/System.out: 1k tbn
03-25 03:08:25.579 3112-3156/com.domain.appname I/System.out: Stream #0:1
03-25 03:08:25.579 3112-3156/com.domain.appname I/System.out: Stream #0:1
03-25 03:08:25.579 3112-3156/com.domain.appname I/System.out: , 0, 1/1000
03-25 03:08:25.579 3112-3156/com.domain.appname I/System.out: : Audio: aac (LC) ([10][0][0][0] / 0x000A), 44100 Hz, stereo, fltp, delay 1024, 128 kb/s
03-25 03:08:25.579 3112-3156/com.domain.appname I/System.out: : Audio: aac (LC) ([10][0][0][0] / 0x000A), 44100 Hz, stereo, fltp, delay 1024, 128 kb/s
03-25 03:08:25.604 3112-3215/com.domain.appname I/System.out: [SWR @ 0xdfa89000] Using s16p internally between filters
03-25 03:08:25.677 3112-3216/com.domain.appname I/System.out: [in @ 0xe86adb40] Changing frame properties on the fly is not supported by all filters.
03-25 03:08:25.677 3112-3216/com.domain.appname I/System.out: [in @ 0xe86adb40] Changing frame properties on the fly is not supported by all filters.
03-25 03:08:25.678 3112-3216/com.domain.appname I/System.out: [auto_scaler_0 @ 0xe86adcc0] w:1280 h:720 fmt:nv21 sar:0/1 -> w:1280 h:720 fmt:yuva420p sar:0/1 flags:0x2
03-25 03:08:25.716 3112-3216/com.domain.appname I/System.out: [in @ 0xe86adb40] Changing frame properties on the fly is not supported by all filters.
03-25 03:08:25.716 3112-3216/com.domain.appname I/System.out: [in @ 0xe86adb40] Changing frame properties on the fly is not supported by all filters.
03-25 03:08:25.770 3112-3216/com.domain.appname I/System.out: [in @ 0xe86adb40] Changing frame properties on the fly is not supported by all filters.
03-25 03:08:25.770 3112-3216/com.domain.appname I/System.out: [in @ 0xe86adb40] Changing frame properties on the fly is not supported by all filters.
...
Where the first part is initialization, and the second are two frames.
DeKinci
on 25 Mar 2018
It says something about scaling. Maybe there is a bug in there, so try to
use an image that has the same size as the video.
saudet
on 25 Mar 2018
Nothing changed
DeKinci
on 25 Mar 2018
Maybe this issue is caused by the same reason as in #232, in the last comment, also it was mentioned in #507. There is nothing about the solution, but it may help to find where the bug appears.
DeKinci
on 25 Mar 2018
Ok, maybe it doesn't support the pixel format of the video or of the image.
Try to convert both of them to some other pixel format, but the same for
both.
saudet
on 26 Mar 2018
I'm using yuv420, it's sad if it is not supported
DeKinci
on 26 Mar 2018
Like I said try to use a filter to convert it. The command line tool may be
doing that automatically.
saudet
on 26 Mar 2018
I used rgb24 and it didn't helped
DeKinci
on 26 Mar 2018
Ok, please post the whole filter command you used and the whole log output again.
saudet
on 26 Mar 2018
Ah, I might have an answer here:
loop
Specifies how many times to read the stream in sequence. If the value is 0, the stream will be looped infinitely. Default value is "1".
https://ffmpeg.org/ffmpeg-filters.html#movie-1
We should probably use 0 for an image file.
saudet
on 26 Mar 2018
I've tried that already and nothing changed. Also, it works without loop argument on a desktop version
DeKinci
on 26 Mar 2018
So it works with JavaCV on Windows, just not on Android? That sounds like a permission issue...
saudet
on 26 Mar 2018
No, it works fine on ffmpeg. I didn't try it on a desktop JavaCV, but according to other comments it is not the case.
DeKinci
on 26 Mar 2018
This is a new filter command:
filter = new FFmpegFrameFilter(
"movie=" + path + ":format_name=rgb24[img];" +
"[in]format=pix_fmts=rgb24[fin];" +
"[fin][img]overlay=format=rgb[out]",
imageWidth, imageHeight);
And this is log for it:
detected 4 logical cores
[in @ 0xd86ea600] Setting 'video_size' to value '1280x720'
[in @ 0xd86ea600] Setting 'pix_fmt' to value '0'
[in @ 0xd86ea600] Setting 'time_base' to value '1/30'
[in @ 0xd86ea600] Setting 'pixel_aspect' to value '1/1'
[in @ 0xd86ea600] w:1280 h:720 pixfmt:yuv420p tb:1/30 fr:0/1 sar:1/1 sws_param:
[Parsed_movie_0 @ 0xd86ea7e0] Setting 'filename' to value '/storage/emulated/0/AppName/img.png'
[Parsed_movie_0 @ 0xd86ea7e0] Setting 'format_name' to value 'rgb24'
[NULL @ 0xd9274200] Opening '/storage/emulated/0/AppName/img.png' for reading
[file @ 0xd9b70690] Setting default whitelist 'file,crypto'
[png_pipe @ 0xd9274200] Format png_pipe probed with size=2048 and score=99
[png_pipe @ 0xd9274200] Before avformat_find_stream_info() pos: 0 bytes read:32768 seeks:0 nb_streams:1
[png_pipe @ 0xd9274200] After avformat_find_stream_info() pos: 41290 bytes read:41290 seeks:0 frames:1
[Parsed_movie_0 @ 0xd86ea7e0] seek_point:0 format_name:rgb24 file_name:/storage/emulated/0/AppName/img.png stream_index:-1
[Parsed_format_1 @ 0xd86ea840] compat: called with args=[pix_fmts=rgb24]
[Parsed_format_1 @ 0xd86ea840] Setting 'pix_fmts' to value 'rgb24'
[Parsed_overlay_2 @ 0xd86ea8a0] Setting 'format' to value 'rgb'
[auto_scaler_0 @ 0xd86ea900] w:iw h:ih flags:'bilinear' interl:0
[Parsed_format_1 @ 0xd86ea840] auto-inserting filter 'auto_scaler_0' between the filter 'in' and the filter 'Parsed_format_1'
[AVFilterGraph @ 0xdcf6d2c0] query_formats: 5 queried, 3 merged, 1 already done, 0 delayed
[auto_scaler_0 @ 0xd86ea900] w:1280 h:720 fmt:yuv420p sar:1/1 -> w:1280 h:720 fmt:rgb24 sar:1/1 flags:0x2
[Parsed_overlay_2 @ 0xd86ea8a0] main w:1280 h:720 fmt:rgb24 overlay w:1280 h:720 fmt:rgba
[Parsed_overlay_2 @ 0xd86ea8a0] [framesync @ 0xe8f4e420] Selected 1/150 time base
[Parsed_overlay_2 @ 0xd86ea8a0] [framesync @ 0xe8f4e420] Sync level 2
[libx264 @ 0xdd152c00] using cpu capabilities: none!
[libx264 @ 0xdd152c00] using cpu capabilities: none!
[libx264 @ 0xdd152c00] profile Constrained Baseline, level 3.1
[libx264 @ 0xdd152c00] profile Constrained Baseline, level 3.1
[libx264 @ 0xdd152c00] 264 - core 152 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=4 lookahead_threads=4 sliced_threads=1 slices=4 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=60 keyint_min=6 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=27.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
[libx264 @ 0xdd152c00] 264 - core 152 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=4 lookahead_threads=4 sliced_threads=1 slices=4 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=60 keyint_min=6 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=27.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
[rtmp @ 0xd9b70770] No default whitelist set
[tcp @ 0xd86eade0] No default whitelist set
[rtmp @ 0xd9b70770] Handshaking...
[rtmp @ 0xd9b70770] Type answer 3
[rtmp @ 0xd9b70770] Server version 4.0.0.1
[rtmp @ 0xd9b70770] Proto = rtmp, path = /live2/vtmb-yjbw-4hpr-1jjw, app = live2, fname = vtmb-yjbw-4hpr-1jjw
[rtmp @ 0xd9b70770] Window acknowledgement size = 2500000
[rtmp @ 0xd9b70770] Max sent, unacked = 10000000
[rtmp @ 0xd9b70770] New incoming chunk size = 256
[rtmp @ 0xd9b70770] Releasing stream...
[rtmp @ 0xd9b70770] FCPublish stream...
[rtmp @ 0xd9b70770] Creating stream...
[rtmp @ 0xd9b70770] Sending publish command for 'vtmb-yjbw-4hpr-1jjw'
Output #0, flv, to 'rtmp://a.rtmp.youtube.com/live2/vtmb-yjbw-4hpr-1jjw':
Output #0, flv, to 'rtmp://a.rtmp.youtube.com/live2/vtmb-yjbw-4hpr-1jjw':
Metadata:
Metadata:
encoder :
encoder :
Lavf57.83.100
Lavf57.83.100
Stream #0:0
Stream #0:0
, 0, 1/1000
: Video: h264 (Constrained Baseline), 1 reference frame ([7][0][0][0] / 0x0007), yuv420p, 1280x720 (0x0), 0/1, q=2-31, 2764 kb/s
: Video: h264 (Constrained Baseline), 1 reference frame ([7][0][0][0] / 0x0007), yuv420p, 1280x720 (0x0), 0/1, q=2-31, 2764 kb/s
,
,
1k tbn
1k tbn
Stream #0:1
Stream #0:1
, 0, 1/1000
: Audio: aac (LC) ([10][0][0][0] / 0x000A), 44100 Hz, stereo, fltp, delay 1024, 128 kb/s
: Audio: aac (LC) ([10][0][0][0] / 0x000A), 44100 Hz, stereo, fltp, delay 1024, 128 kb/s
[in @ 0xd86ea600] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd86ea600] Changing frame properties on the fly is not supported by all filters.
[auto_scaler_0 @ 0xd86ea900] w:1280 h:720 fmt:nv21 sar:0/1 -> w:1280 h:720 fmt:rgb24 sar:0/1 flags:0x2
[SWR @ 0xdf1e4000] Using s16p internally between filters
[libx264 @ 0xdd152c00] frame= 0 QP=24.00 NAL=3 Slice:I Poc:0 I:3600 P:0 SKIP:0 size=3929 bytes
[in @ 0xd86ea600] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd86ea600] Changing frame properties on the fly is not supported by all filters.
[libx264 @ 0xdd152c00] frame= 1 QP=20.00 NAL=2 Slice:P Poc:2 I:5 P:15 SKIP:3580 size=471 bytes
[in @ 0xd86ea600] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd86ea600] Changing frame properties on the fly is not supported by all filters.
[libx264 @ 0xdd152c00] frame= 2 QP=20.00 NAL=2 Slice:P Poc:4 I:5 P:12 SKIP:3583 size=145 bytes
[in @ 0xd86ea600] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd86ea600] Changing frame properties on the fly is not supported by all filters.
[libx264 @ 0xdd152c00] frame= 3 QP=19.00 NAL=2 Slice:P Poc:6 I:4 P:18 SKIP:3578 size=845 bytes
[in @ 0xd86ea600] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd86ea600] Changing frame properties on the fly is not supported by all filters.
An image scaler is still being inserted, that's strange.
Anyway, if we compare that with a log of the ffmpeg program, I'm sure we'll find the difference pretty quickly. What does the output for the ffmpeg program look like at the same log level?
saudet
on 26 Mar 2018
The command is ffmpeg -loglevel debug -i video.mp4 -vf movie=img.png[img];[in][img]overlay[out] output.mp4 (I removed changing formats because it doesn't help and the PC player refused to play video in new format), also the image size is same as the video size.
And the log is
ffmpeg version 3.4.2 Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 7.3.0 (GCC)
configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libmfx --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth
libavutil 55. 78.100 / 55. 78.100
libavcodec 57.107.100 / 57.107.100
libavformat 57. 83.100 / 57. 83.100
libavdevice 57. 10.100 / 57. 10.100
libavfilter 6.107.100 / 6.107.100
libswscale 4. 8.100 / 4. 8.100
libswresample 2. 9.100 / 2. 9.100
libpostproc 54. 7.100 / 54. 7.100
Splitting the commandline.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'.
Reading option '-i' ... matched as input url with argument 'video.mp4'.
Reading option '-vf' ... matched as option 'vf' (set video filters) with argument 'movie=img.png[img];[in][img]overlay[out]'.
Reading option 'output.mp4' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option loglevel (set logging level) with argument debug.
Successfully parsed a group of options.
Parsing a group of options: input url video.mp4.
Successfully parsed a group of options.
Opening an input file: video.mp4.
[NULL @ 000001b8977e24a0] Opening 'video.mp4' for reading
[file @ 000001b8978d6dc0] Setting default whitelist 'file,crypto'
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001b8977e24a0] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001b8977e24a0] ISO: File Type Major Brand: isom
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001b8977e24a0] Unknown dref type 0x206c7275 size 12
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001b8977e24a0] Processing st: 0, edit list 0 - media time: 0, duration: 2186752
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001b8977e24a0] Unknown dref type 0x206c7275 size 12
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001b8977e24a0] Processing st: 1, edit list 0 - media time: 0, duration: 8201232
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001b8977e24a0] Before avformat_find_stream_info() pos: 31491130 bytes read:130297 seeks:1 nb_streams:2
[h264 @ 000001b8977f4560] nal_unit_type: 7, nal_ref_idc: 3
[h264 @ 000001b8977f4560] nal_unit_type: 8, nal_ref_idc: 3
[h264 @ 000001b8977f4560] nal_unit_type: 5, nal_ref_idc: 3
[h264 @ 000001b8977f4560] Reinit context to 1280x720, pix_fmt: yuv420p
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001b8977e24a0] All info found
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001b8977e24a0] After avformat_find_stream_info() pos: 106237 bytes read:236486 seeks:2 frames:2
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'video.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
creation_time : 1970-01-01T00:00:00.000000Z
encoder : Lavf53.24.2
Duration: 00:02:50.86, start: 0.000000, bitrate: 1474 kb/s
Stream #0:0(und), 1, 1/12800: Video: h264 (Main), 1 reference frame (avc1 / 0x31637661), yuv420p(left), 1280x720 [SAR 1:1 DAR 16:9], 0/1, 1086 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
Metadata:
creation_time : 1970-01-01T00:00:00.000000Z
handler_name : VideoHandler
Stream #0:1(und), 1, 1/48000: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 5.1, fltp, 383 kb/s (default)
Metadata:
creation_time : 1970-01-01T00:00:00.000000Z
handler_name : SoundHandler
Successfully opened the file.
Parsing a group of options: output url output.mp4.
Applying option vf (set video filters) with argument movie=img.png[img];[in][img]overlay[out].
Successfully parsed a group of options.
Opening an output file: output.mp4.
[file @ 000001b8978dd440] Setting default whitelist 'file,crypto'
Successfully opened the file.
detected 8 logical cores
[h264 @ 000001b8978de6c0] nal_unit_type: 7, nal_ref_idc: 3
[h264 @ 000001b8978de6c0] nal_unit_type: 8, nal_ref_idc: 3
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[graph_1_in_0_1 @ 000001b89942d880] Setting 'time_base' to value '1/48000'
[graph_1_in_0_1 @ 000001b89942d880] Setting 'sample_rate' to value '48000'
[graph_1_in_0_1 @ 000001b89942d880] Setting 'sample_fmt' to value 'fltp'
[graph_1_in_0_1 @ 000001b89942d880] Setting 'channel_layout' to value '0x3f'
[graph_1_in_0_1 @ 000001b89942d880] tb:1/48000 samplefmt:fltp samplerate:48000 chlayout:0x3f
[format_out_0_1 @ 000001b89972bc80] Setting 'sample_fmts' to value 'fltp'
[format_out_0_1 @ 000001b89972bc80] Setting 'sample_rates' to value '96000|88200|64000|48000|44100|32000|24000|22050|16000|12000|11025|8000|7350'
[AVFilterGraph @ 000001b89968b240] query_formats: 4 queried, 9 merged, 0 already done, 0 delayed
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Last message repeated 1 times
[h264 @ 000001b8978de6c0] nal_unit_type: 5, nal_ref_idc: 3
[h264 @ 000001b8978de6c0] Reinit context to 1280x720, pix_fmt: yuv420p
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Last message repeated 2 times
[h264 @ 000001b89780fa20] nal_unit_type: 1, nal_ref_idc: 2
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Last message repeated 2 times
[h264 @ 000001b8978101c0] nal_unit_type: 1, nal_ref_idc: 2
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Last message repeated 2 times
[h264 @ 000001b897811040] nal_unit_type: 1, nal_ref_idc: 2
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Last message repeated 2 times
[h264 @ 000001b89780ae20] nal_unit_type: 1, nal_ref_idc: 2
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Last message repeated 2 times
[h264 @ 000001b89787f000] nal_unit_type: 1, nal_ref_idc: 2
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Last message repeated 2 times
[h264 @ 000001b89787fe80] nal_unit_type: 1, nal_ref_idc: 2
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Last message repeated 1 times
[h264 @ 000001b8978811c0] nal_unit_type: 1, nal_ref_idc: 2
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Last message repeated 1 times
[h264 @ 000001b89787eb40] nal_unit_type: 1, nal_ref_idc: 2
[Parsed_movie_0 @ 000001b899616f20] Setting 'filename' to value 'img.png'
[NULL @ 000001b89b1017e0] Opening 'img.png' for reading
[file @ 000001b899bc6b60] Setting default whitelist 'file,crypto'
[png_pipe @ 000001b89b1017e0] Format png_pipe probed with size=2048 and score=99
[png_pipe @ 000001b89b1017e0] Before avformat_find_stream_info() pos: 0 bytes read:32768 seeks:0 nb_streams:1
[png_pipe @ 000001b89b1017e0] After avformat_find_stream_info() pos: 41290 bytes read:41290 seeks:0 frames:1
[Parsed_movie_0 @ 000001b899616f20] seek_point:0 format_name:(null) file_name:img.png stream_index:-1
[graph 0 input from stream 0:0 @ 000001b89b151740] Setting 'video_size' to value '1280x720'
[graph 0 input from stream 0:0 @ 000001b89b151740] Setting 'pix_fmt' to value '0'
[graph 0 input from stream 0:0 @ 000001b89b151740] Setting 'time_base' to value '1/12800'
[graph 0 input from stream 0:0 @ 000001b89b151740] Setting 'pixel_aspect' to value '1/1'
[graph 0 input from stream 0:0 @ 000001b89b151740] Setting 'sws_param' to value 'flags=2'
[graph 0 input from stream 0:0 @ 000001b89b151740] Setting 'frame_rate' to value '25/1'
[graph 0 input from stream 0:0 @ 000001b89b151740] w:1280 h:720 pixfmt:yuv420p tb:1/12800 fr:25/1 sar:1/1 sws_param:flags=2
[format @ 000001b89b151340] compat: called with args=[yuv420p|yuvj420p|yuv422p|yuvj422p|yuv444p|yuvj444p|yuv420p10le|yuv422p10le|yuv444p10le|nv12|nv16|nv20le|nv21]
[format @ 000001b89b151340] Setting 'pix_fmts' to value 'yuv420p|yuvj420p|yuv422p|yuvj422p|yuv444p|yuvj444p|yuv420p10le|yuv422p10le|yuv444p10le|nv12|nv16|nv20le|nv21'
[auto_scaler_0 @ 000001b89b151f60] Setting 'flags' to value 'bicubic'
[auto_scaler_0 @ 000001b89b151f60] w:iw h:ih flags:'bicubic' interl:0
[Parsed_overlay_1 @ 000001b89b151680] auto-inserting filter 'auto_scaler_0' between the filter 'Parsed_movie_0' and the filter 'Parsed_overlay_1'
[AVFilterGraph @ 000001b89a00b6c0] query_formats: 5 queried, 3 merged, 1 already done, 0 delayed
[auto_scaler_0 @ 000001b89b151f60] w:1280 h:720 fmt:rgba sar:1/1 -> w:1280 h:720 fmt:yuva420p sar:1/1 flags:0x4
[Parsed_overlay_1 @ 000001b89b151680] main w:1280 h:720 fmt:yuv420p overlay w:1280 h:720 fmt:yuva420p
[Parsed_overlay_1 @ 000001b89b151680] [framesync @ 000001b8978dcf68] Selected 1/12800 time base
[Parsed_overlay_1 @ 000001b89b151680] [framesync @ 000001b8978dcf68] Sync level 2
[auto_scaler_0 @ 000001b89b151f60] w:1280 h:720 fmt:rgba sar:2835/2835 -> w:1280 h:720 fmt:yuva420p sar:1/1 flags:0x4
[Parsed_overlay_1 @ 000001b89b151680] n:1.000000 t:0.000000 pos:1015.000000 x:0.000000 xi:0 y:0.000000 yi:0
[libx264 @ 000001b899284360] using mv_range_thread = 24
[libx264 @ 000001b899284360] using SAR=1/1
[libx264 @ 000001b899284360] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 000001b899284360] profile High, level 3.1
[libx264 @ 000001b899284360] 264 - core 155 r2901 7d0ff22 - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'output.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf57.83.100
Stream #0:0(und), 0, 1/12800: Video: h264 (libx264), 1 reference frame (avc1 / 0x31637661), yuv420p(left), 1280x720 [SAR 1:1 DAR 16:9], 0/1, q=-1--1, 25 fps, 12800 tbn, 25 tbc (default)
Metadata:
creation_time : 1970-01-01T00:00:00.000000Z
handler_name : VideoHandler
encoder : Lavc57.107.100 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
Stream #0:1(und), 0, 1/48000: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 5.1, fltp, delay 1024, 341 kb/s (default)
Metadata:
creation_time : 1970-01-01T00:00:00.000000Z
handler_name : SoundHandler
encoder : Lavc57.107.100 aac
Clipping frame in rate conversion by 0.000008
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Last message repeated 1 times
[Parsed_movie_0 @ 000001b899616f20] EOF timestamp not reliable
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[Parsed_overlay_1 @ 000001b89b151680] Copying data in avfilter.
[h264 @ 000001b8978de6c0] nal_unit_type: 1, nal_ref_idc: 2
[Parsed_overlay_1 @ 000001b89b151680] n:2.000000 t:0.040000 pos:108274.000000 x:0.000000 xi:0 y:0.000000 yi:0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Last message repeated 2 times
[Parsed_overlay_1 @ 000001b89b151680] Copying data in avfilter.
Last message repeated 2 times
[h264 @ 000001b89780fa20] nal_unit_type: 1, nal_ref_idc: 2
[Parsed_overlay_1 @ 000001b89b151680] n:3.000000 t:0.080000 pos:111848.000000 x:0.000000 xi:0 y:0.000000 yi:0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Last message repeated 2 times
[Parsed_overlay_1 @ 000001b89b151680] Copying data in avfilter.
Last message repeated 2 times
[h264 @ 000001b8978101c0] nal_unit_type: 1, nal_ref_idc: 2
[Parsed_overlay_1 @ 000001b89b151680] n:4.000000 t:0.120000 pos:115932.000000 x:0.000000 xi:0 y:0.000000 yi:0
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
Last message repeated 2 times
...
...
...
[Parsed_overlay_1 @ 000001b89b151680] n:4271.000000 t:170.800000 pos:31384220.000000 x:0.000000 xi:0 y:0.000000 yi:0
[libx264 @ 000001b899284360] frame=4214 QP=24.15 NAL=0 Slice:B Poc:120 I:0 P:411 SKIP:3189 size=496 bytes
[Parsed_overlay_1 @ 000001b89b151680] [framesync @ 000001b8978dcf68] Sync level 0
[out_0_0 @ 000001b89b151d00] EOF on sink link out_0_0:default.
[out_0_1 @ 000001b89942da00] EOF on sink link out_0_1:default.
No more output streams to write to, finishing.
[libx264 @ 000001b899284360] frame=4215 QP=23.77 NAL=0 Slice:B Poc:124 I:0 P:462 SKIP:3137 size=514 bytes
[libx264 @ 000001b899284360] frame=4216 QP=21.05 NAL=2 Slice:P Poc:134 I:134 P:1049 SKIP:2417 size=5719 bytes
...
...
frame= 4271 fps=112 q=-1.0 Lsize= 29425kB time=00:02:50.85 bitrate=1410.8kbits/s speed=4.46x
video:22182kB audio:7125kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.400701%
Input file #0 (video.mp4):
Input stream #0:0 (video): 4271 packets read (23193765 bytes); 4271 frames decoded;
Input stream #0:1 (audio): 8009 packets read (8199788 bytes); 8009 frames decoded (8201216 samples);
Total: 12280 packets (31393553 bytes) demuxed
Output file #0 (output.mp4):
Output stream #0:0 (video): 4271 frames encoded; 4271 packets muxed (22714566 bytes);
Output stream #0:1 (audio): 8009 frames encoded (8201216 samples); 8010 packets muxed (7296351 bytes);
Total: 12281 packets (30010917 bytes) muxed
12280 frames successfully decoded, 0 decoding errors
[AVIOContext @ 000001b89982ee00] Statistics: 41290 bytes read, 0 seeks
[AVIOContext @ 000001b8978dd4e0] Statistics: 2 seeks, 118 writeouts
[libx264 @ 000001b899284360] frame I:44 Avg QP:15.98 size: 88754
[libx264 @ 000001b899284360] frame P:2394 Avg QP:21.02 size: 6501
[libx264 @ 000001b899284360] frame B:1833 Avg QP:25.25 size: 1770
[libx264 @ 000001b899284360] consecutive B-frames: 37.8% 11.5% 10.1% 40.6%
[libx264 @ 000001b899284360] mb I I16..4: 18.1% 37.3% 44.6%
[libx264 @ 000001b899284360] mb P I16..4: 1.4% 3.2% 0.4% P16..4: 22.8% 5.4% 2.4% 0.0% 0.0% skip:64.4%
[libx264 @ 000001b899284360] mb B I16..4: 0.1% 0.5% 0.1% B16..8: 17.8% 1.2% 0.2% direct: 0.5% skip:79.6% L0:47.4% L1:50.0% BI: 2.6%
[libx264 @ 000001b899284360] 8x8 transform intra:57.5% inter:68.6%
[libx264 @ 000001b899284360] coded y,uvDC,uvAC intra: 54.0% 73.5% 28.9% inter: 5.3% 8.2% 0.4%
[libx264 @ 000001b899284360] i16 v,h,dc,p: 24% 31% 12% 32%
[libx264 @ 000001b899284360] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 24% 20% 20% 5% 6% 7% 6% 6% 6%
[libx264 @ 000001b899284360] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 26% 17% 11% 6% 8% 9% 7% 8% 6%
[libx264 @ 000001b899284360] i8c dc,h,v,p: 43% 25% 21% 11%
[libx264 @ 000001b899284360] Weighted P-Frames: Y:0.1% UV:0.1%
[libx264 @ 000001b899284360] ref P L0: 79.0% 9.6% 8.8% 2.6% 0.0%
[libx264 @ 000001b899284360] ref B L0: 92.7% 6.3% 0.9%
[libx264 @ 000001b899284360] ref B L1: 97.1% 2.9%
[libx264 @ 000001b899284360] kb/s:1063.63
[aac @ 000001b8977e6520] Qavg: 545.756
[AVIOContext @ 000001b8977eb6e0] Statistics: 31544589 bytes read, 2 seeks
I included full log replacing repetetive parts with ellipses
DeKinci
on 27 Mar 2018
With FFmpegFrameFilter you get:
[in @ 0xd86ea600] Setting 'time_base' to value '1/30'
...
[Parsed_overlay_2 @ 0xd86ea8a0] [framesync @ 0xe8f4e420] Selected 1/150 time base
And nothing else about "Parsed_overlay_2". But with the ffmpeg program you are getting:
[graph 0 input from stream 0:0 @ 000001b89b151740] Setting 'time_base' to value '1/12800'
...
[Parsed_overlay_1 @ 000001b89b151680] [framesync @ 000001b8978dcf68] Selected 1/12800 time base
[Parsed_overlay_1 @ 000001b89b151680] n:1.000000 t:0.000000 pos:1015.000000 x:0.000000 xi:0 y:0.000000 yi:0
That looks suspicious... Maybe try to use the same time_base with FFmpegFrameFilter as well?
There might be a "setpts=N" missing in there too: https://github.com/bytedeco/javacv/issues/928#issuecomment-368438201
saudet
on 28 Mar 2018
Where should I apply setpts filter? To the overlay or to the main stream, maybe to both of them or to the result?
How to specify time_base in FFmpegFrameFilter? Documintation says it is a buffer argument, but how to get access to the buffer?
DeKinci
on 29 Mar 2018
Are you saying that you actually tried to put setpts on all three of them and it didn't work?
The time_base is based on the frame rate, so try to set it with setFrameRate() before calling start().
saudet
on 29 Mar 2018
Nothing helped.
When I try to use setpts on the video stream it crashes with the following error:
detected 4 logical cores
[in @ 0xd8f3e860] Setting 'video_size' to value '1280x720'
[in @ 0xd8f3e860] Setting 'pix_fmt' to value '0'
[in @ 0xd8f3e860] Setting 'time_base' to value '1/30'
[in @ 0xd8f3e860] Setting 'pixel_aspect' to value '1/1'
[in @ 0xd8f3e860] w:1280 h:720 pixfmt:yuv420p tb:1/30 fr:0/1 sar:1/1 sws_param:
[Parsed_movie_0 @ 0xd8f3ea40] Setting 'filename' to value '/storage/emulated/0/AppName/img.png'
[NULL @ 0xe0e42000] Opening '/storage/emulated/0/AppName/img.png' for reading
[file @ 0xd99025b0] Setting default whitelist 'file,crypto'
[png_pipe @ 0xe0e42000] Format png_pipe probed with size=2048 and score=99
[png_pipe @ 0xe0e42000] Before avformat_find_stream_info() pos: 0 bytes read:32768 seeks:0 nb_streams:1
[png_pipe @ 0xe0e42000] After avformat_find_stream_info() pos: 179559 bytes read:179559 seeks:0 frames:1
[Parsed_movie_0 @ 0xd8f3ea40] seek_point:0 format_name:(null) file_name:/storage/emulated/0/AppName/img.png stream_index:-1
[Parsed_setpts_1 @ 0xd8f3eb60] Setting 'expr' to value 'N/(30*TB)'
[Parsed_setpts_2 @ 0xd8f3eb00] Setting 'expr' to value 'N/(30*TB)'
[Parsed_setpts_4 @ 0xd8f3ebc0] Setting 'expr' to value 'N/(30*TB)'
[auto_scaler_0 @ 0xd8f3ec20] w:iw h:ih flags:'bilinear' interl:0
[Parsed_overlay_3 @ 0xd8f3eaa0] auto-inserting filter 'auto_scaler_0' between the filter 'Parsed_setpts_1' and the filter 'Parsed_overlay_3'
[AVFilterGraph @ 0xd8f10040] query_formats: 7 queried, 5 merged, 1 already done, 0 delayed
[in @ 0xd8f3e860] TB:0.033333 FRAME_RATE:nan SAMPLE_RATE:nan
[Parsed_movie_0 @ 0xd8f3ea40] TB:0.040000 FRAME_RATE:25.000000 SAMPLE_RATE:nan
[auto_scaler_0 @ 0xd8f3ec20] w:800 h:600 fmt:rgba sar:1/1 -> w:800 h:600 fmt:yuva420p sar:1/1 flags:0x2
[Parsed_overlay_3 @ 0xd8f3eaa0] main w:1280 h:720 fmt:yuv420p overlay w:800 h:600 fmt:yuva420p
[Parsed_overlay_3 @ 0xd8f3eaa0] [framesync @ 0xf0404d20] Selected 1/150 time base
[Parsed_overlay_3 @ 0xd8f3eaa0] [framesync @ 0xf0404d20] Sync level 2
[Parsed_overlay_3 @ 0xd8f3eaa0] TB:0.033333 FRAME_RATE:nan SAMPLE_RATE:nan
[in @ 0xd8f3e860] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd8f3e860] Changing frame properties on the fly is not supported by all filters.
[auto_scaler_0 @ 0xd8f3ec20] w:800 h:600 fmt:rgba sar:0/1 -> w:800 h:600 fmt:yuva420p sar:0/1 flags:0x2
[Parsed_overlay_3 @ 0xd8f3eaa0] n:1.000000 t:0.000000 pos:nan x:0.000000 xi:0 y:0.000000 yi:0
Fatal signal 11 (SIGSEGV), code 1, fault addr 0x0 in tid 7510 (pool-10-thread-)
Otherwise nothing changes: works fine, no image. I tried applying setpts to image and output and simultaneously and also I used setFrameRate().
The log with setpts:
detected 4 logical cores
[in @ 0xd92d5c60] Setting 'video_size' to value '1280x720'
[in @ 0xd92d5c60] Setting 'pix_fmt' to value '0'
[in @ 0xd92d5c60] Setting 'time_base' to value '1/30'
[in @ 0xd92d5c60] Setting 'pixel_aspect' to value '1/1'
[in @ 0xd92d5c60] w:1280 h:720 pixfmt:yuv420p tb:1/30 fr:0/1 sar:1/1 sws_param:
[Parsed_movie_0 @ 0xd92d5ea0] Setting 'filename' to value '/storage/emulated/0/AppName/img.png'
[NULL @ 0xd92c7e00] Opening '/storage/emulated/0/AppName/img.png' for reading
[file @ 0xd9b82700] Setting default whitelist 'file,crypto'
[png_pipe @ 0xd92c7e00] Format png_pipe probed with size=2048 and score=99
[png_pipe @ 0xd92c7e00] Before avformat_find_stream_info() pos: 0 bytes read:32768 seeks:0 nb_streams:1
[png_pipe @ 0xd92c7e00] After avformat_find_stream_info() pos: 179559 bytes read:179559 seeks:0 frames:1
[Parsed_movie_0 @ 0xd92d5ea0] seek_point:0 format_name:(null) file_name:/storage/emulated/0/AppName/img.png stream_index:-1
[Parsed_setpts_1 @ 0xd92d5f00] Setting 'expr' to value 'N/(30*TB)'
[Parsed_setpts_3 @ 0xd92d5fc0] Setting 'expr' to value 'N/(30*TB)'
[auto_scaler_0 @ 0xd92d6200] w:iw h:ih flags:'bilinear' interl:0
[Parsed_overlay_2 @ 0xd92d5f60] auto-inserting filter 'auto_scaler_0' between the filter 'Parsed_setpts_1' and the filter 'Parsed_overlay_2'
[AVFilterGraph @ 0xda24f5c0] query_formats: 6 queried, 4 merged, 1 already done, 0 delayed
[Parsed_movie_0 @ 0xd92d5ea0] TB:0.040000 FRAME_RATE:25.000000 SAMPLE_RATE:nan
[auto_scaler_0 @ 0xd92d6200] w:800 h:600 fmt:rgba sar:1/1 -> w:800 h:600 fmt:yuva420p sar:1/1 flags:0x2
[Parsed_overlay_2 @ 0xd92d5f60] main w:1280 h:720 fmt:yuv420p overlay w:800 h:600 fmt:yuva420p
[Parsed_overlay_2 @ 0xd92d5f60] [framesync @ 0xe0fe1ee0] Selected 1/150 time base
[Parsed_overlay_2 @ 0xd92d5f60] [framesync @ 0xe0fe1ee0] Sync level 2
[Parsed_overlay_2 @ 0xd92d5f60] TB:0.033333 FRAME_RATE:nan SAMPLE_RATE:nan
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[auto_scaler_0 @ 0xd92d6200] w:800 h:600 fmt:rgba sar:0/1 -> w:800 h:600 fmt:yuva420p sar:0/1 flags:0x2
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[libx264 @ 0xd7c3e400] using cpu capabilities: none!
[libx264 @ 0xd7c3e400] using cpu capabilities: none!
[libx264 @ 0xd7c3e400] profile Constrained Baseline, level 3.1
[libx264 @ 0xd7c3e400] profile Constrained Baseline, level 3.1
[libx264 @ 0xd7c3e400] 264 - core 152 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=4 lookahead_threads=4 sliced_threads=1 slices=4 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=60 keyint_min=6 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=27.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
[libx264 @ 0xd7c3e400] 264 - core 152 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=4 lookahead_threads=4 sliced_threads=1 slices=4 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=60 keyint_min=6 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=27.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[rtmp @ 0xd9b825b0] No default whitelist set
[tcp @ 0xd6f587e0] No default whitelist set
[rtmp @ 0xd9b825b0] Handshaking...
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[rtmp @ 0xd9b825b0] Type answer 3
[rtmp @ 0xd9b825b0] Server version 4.0.0.1
[rtmp @ 0xd9b825b0] Proto = rtmp, path = /live2/9bgh-jjfr-7k4x-cr0d, app = live2, fname = 9bgh-jjfr-7k4x-cr0d
[rtmp @ 0xd9b825b0] Window acknowledgement size = 2500000
[rtmp @ 0xd9b825b0] Max sent, unacked = 10000000
[rtmp @ 0xd9b825b0] New incoming chunk size = 256
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[rtmp @ 0xd9b825b0] Releasing stream...
[rtmp @ 0xd9b825b0] FCPublish stream...
[rtmp @ 0xd9b825b0] Creating stream...
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[rtmp @ 0xd9b825b0] Sending publish command for '9bgh-jjfr-7k4x-cr0d'
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
Output #0, flv, to 'rtmp://a.rtmp.youtube.com/live2/9bgh-jjfr-7k4x-cr0d':
Output #0, flv, to 'rtmp://a.rtmp.youtube.com/live2/9bgh-jjfr-7k4x-cr0d':
Metadata:
Metadata:
encoder :
encoder :
Lavf57.83.100
Lavf57.83.100
Stream #0:0
Stream #0:0
, 0, 1/1000
: Video: h264 (Constrained Baseline), 1 reference frame ([7][0][0][0] / 0x0007), yuv420p, 1280x720 (0x0), 0/1, q=2-31, 2764 kb/s
: Video: h264 (Constrained Baseline), 1 reference frame ([7][0][0][0] / 0x0007), yuv420p, 1280x720 (0x0), 0/1, q=2-31, 2764 kb/s
,
,
1k tbn
1k tbn
Stream #0:1
Stream #0:1
, 0, 1/1000
: Audio: aac (LC) ([10][0][0][0] / 0x000A), 44100 Hz, stereo, fltp, delay 1024, 128 kb/s
: Audio: aac (LC) ([10][0][0][0] / 0x000A), 44100 Hz, stereo, fltp, delay 1024, 128 kb/s
[SWR @ 0xd4802000] Using s16p internally between filters
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[libx264 @ 0xd7c3e400] frame= 0 QP=24.00 NAL=3 Slice:I Poc:0 I:3600 P:0 SKIP:0 size=4003 bytes
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[libx264 @ 0xd7c3e400] frame= 1 QP=20.00 NAL=2 Slice:P Poc:2 I:0 P:18 SKIP:3582 size=550 bytes
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[libx264 @ 0xd7c3e400] frame= 2 QP=19.00 NAL=2 Slice:P Poc:4 I:1 P:17 SKIP:3582 size=260 bytes
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[libx264 @ 0xd7c3e400] frame= 3 QP=18.00 NAL=2 Slice:P Poc:6 I:2 P:19 SKIP:3579 size=562 bytes
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[libx264 @ 0xd7c3e400] frame= 4 QP=18.00 NAL=2 Slice:P Poc:8 I:0 P:10 SKIP:3590 size=94 bytes
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd92d5c60] Changing frame properties on the fly is not supported by all filters.
Maybe I should try using an older version? For example one from 2015 and go up to find where it stopped working? Which was last version in 2015?
Is there any other temporary way to overlay image on video? Using Bitmap for every frame seems pretty slow. So maybe use another library to overlay, and javacv to stream until the bug is fixed? Which to use to get stable result?
DeKinci
on 30 Mar 2018
Well, this works just fine here, so it looks like we just need to use setpts=N after all:
FFmpegFrameFilter filter = new FFmpegFrameFilter("setpts=N[in2];movie=path/to/img.jpg[img];[in2][img]overlay", imageWidth, imageHeight);
FYI, I've made changes in commit https://github.com/bytedeco/javacv/commit/35dd7f8134ff4b31e6ff94ab6049f2a13418cc5a to work around that so we no longer have to specify the setpts=N filter manually.
saudet
on 31 Mar 2018
It crashes on filter.pull() when I use setpts=N[in2];movie=path/to/img.jpg[img];[in2][img]overlay
Here is log:
detected 4 logical cores
[in @ 0xd9ad4560] Setting 'video_size' to value '1280x720'
[in @ 0xd9ad4560] Setting 'pix_fmt' to value '0'
[in @ 0xd9ad4560] Setting 'time_base' to value '1/30'
[in @ 0xd9ad4560] Setting 'pixel_aspect' to value '1/1'
[in @ 0xd9ad4560] w:1280 h:720 pixfmt:yuv420p tb:1/30 fr:0/1 sar:1/1 sws_param:
[Parsed_setpts_0 @ 0xd9ad47a0] Setting 'expr' to value 'N'
[Parsed_movie_1 @ 0xd9ad46e0] Setting 'filename' to value '/storage/emulated/0/AppName/img.png'
[NULL @ 0xd9e5d400] Opening '/storage/emulated/0/AppName/img.png' for reading
[file @ 0xd9af72a0] Setting default whitelist 'file,crypto'
[png_pipe @ 0xd9e5d400] Format png_pipe probed with size=2048 and score=99
[png_pipe @ 0xd9e5d400] Before avformat_find_stream_info() pos: 0 bytes read:32768 seeks:0 nb_streams:1
[png_pipe @ 0xd9e5d400] After avformat_find_stream_info() pos: 179559 bytes read:179559 seeks:0 frames:1
[Parsed_movie_1 @ 0xd9ad46e0] seek_point:0 format_name:(null) file_name:/storage/emulated/0/AppName/img.png stream_index:-1
[auto_scaler_0 @ 0xd9ad4740] w:iw h:ih flags:'bilinear' interl:0
[Parsed_overlay_2 @ 0xd9ad4860] auto-inserting filter 'auto_scaler_0' between the filter 'Parsed_movie_1' and the filter 'Parsed_overlay_2'
[AVFilterGraph @ 0xd9e75780] query_formats: 5 queried, 3 merged, 1 already done, 0 delayed
[in @ 0xd9ad4560] TB:0.033333 FRAME_RATE:nan SAMPLE_RATE:nan
[auto_scaler_0 @ 0xd9ad4740] w:800 h:600 fmt:rgba sar:1/1 -> w:800 h:600 fmt:yuva420p sar:1/1 flags:0x2
[Parsed_overlay_2 @ 0xd9ad4860] main w:1280 h:720 fmt:yuv420p overlay w:800 h:600 fmt:yuva420p
[Parsed_overlay_2 @ 0xd9ad4860] [framesync @ 0xd9e76ca0] Selected 1/150 time base
[Parsed_overlay_2 @ 0xd9ad4860] [framesync @ 0xd9e76ca0] Sync level 2
[in @ 0xd9ad4560] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd9ad4560] Changing frame properties on the fly is not supported by all filters.
[auto_scaler_0 @ 0xd9ad4740] w:800 h:600 fmt:rgba sar:0/1 -> w:800 h:600 fmt:yuva420p sar:0/1 flags:0x2
[Parsed_overlay_2 @ 0xd9ad4860] n:1.000000 t:0.000000 pos:nan x:0.000000 xi:0 y:0.000000 yi:0
```
`filter.pull()` called
A/libc: Fatal signal 11 (SIGSEGV), code 1, fault addr 0x0 in tid 5476 (pool-10-thread-)
```
Without this filter works fine
DeKinci
on 1 Apr 2018
What I've found is that it happens on
int ret = avfilter.av_buffersink_get_frame(this.buffersink_ctx, this.filt_frame);
in FFmpegFrameFilter Frame pull()
DeKinci
on 1 Apr 2018
Well, it works fine here. Try to use JavaCV 1.4.2-SNAPSHOT without setting
it manually.
saudet
on 1 Apr 2018
When I try using newer version (snapshot or just 1.4.1) I'm getting this errors:
V/NativeLoader: Loading the library normally failed: java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "vp8_short_fdct4x4_sse2" referenced by "/data/app/com.domain.domain-2/lib/x86/libavcodec.so"...
at java.lang.Runtime.loadLibrary(Runtime.java:372)
at java.lang.System.loadLibrary(System.java:1076)
at com.getkeepsafe.relinker.SystemLibraryLoader.loadLibrary(SystemLibraryLoader.java:24)
at com.getkeepsafe.relinker.ReLinkerInstance.loadLibraryInternal(ReLinkerInstance.java:163)
at com.getkeepsafe.relinker.ReLinkerInstance.loadLibrary(ReLinkerInstance.java:136)
at com.getkeepsafe.relinker.ReLinkerInstance.loadLibrary(ReLinkerInstance.java:92)
at com.domain.domain.model.util.Utils$NativeLoader.Load(Utils.java:69)
at com.domain.domain.model.platform.youtube.PlatformYoutube.<init>(PlatformYoutube.java:35)
at com.domain.domain.model.platform.youtube.PlatformFactoryYoutube.createPlatform(PlatformFactoryYoutube.java:12)
at com.domain.domain.LogInActivity$1.doInBackground(LogInActivity.java:43)
at com.domain.domain.LogInActivity$1.doInBackground(LogInActivity.java:40)
at android.os.AsyncTask$2.call(AsyncTask.java:295)
at java.util.concurrent.FutureTask.run(FutureTask.java:237)
at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:234)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588)
at java.lang.Thread.run(Thread.java:818)
V/NativeLoader: jniavcodec (null) was not loaded normally, re-linking...
V/NativeLoader: Beginning load of avcodec...
V/NativeLoader: Loading the library normally failed: java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "vp8_short_fdct4x4_sse2" referenced by "/data/app/com.domain.domain-2/lib/x86/libavcodec.so"...
at java.lang.Runtime.loadLibrary(Runtime.java:372)
at java.lang.System.loadLibrary(System.java:1076)
at com.getkeepsafe.relinker.SystemLibraryLoader.loadLibrary(SystemLibraryLoader.java:24)
at com.getkeepsafe.relinker.ReLinkerInstance.loadLibraryInternal(ReLinkerInstance.java:163)
at com.getkeepsafe.relinker.ReLinkerInstance.loadLibrary(ReLinkerInstance.java:136)
at com.getkeepsafe.relinker.ReLinkerInstance.loadLibrary(ReLinkerInstance.java:92)
at com.getkeepsafe.relinker.ReLinkerInstance.loadLibraryInternal(ReLinkerInstance.java:189)
at com.getkeepsafe.relinker.ReLinkerInstance.loadLibrary(ReLinkerInstance.java:136)
at com.getkeepsafe.relinker.ReLinkerInstance.loadLibrary(ReLinkerInstance.java:92)
at com.domain.domain.model.util.Utils$NativeLoader.Load(Utils.java:69)
at com.domain.domain.model.platform.youtube.PlatformYoutube.<init>(PlatformYoutube.java:35)
at com.domain.domain.model.platform.youtube.PlatformFactoryYoutube.createPlatform(PlatformFactoryYoutube.java:12)
at com.domain.domain.LogInActivity$1.doInBackground(LogInActivity.java:43)
at com.domain.domain.LogInActivity$1.doInBackground(LogInActivity.java:40)
at android.os.AsyncTask$2.call(AsyncTask.java:295)
at java.util.concurrent.FutureTask.run(FutureTask.java:237)
at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:234)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588)
at java.lang.Thread.run(Thread.java:818)
V/NativeLoader: avcodec (null) was not loaded normally, re-linking...
V/NativeLoader: Beginning load of swresample...
V/NativeLoader: swresample (null) was loaded normally!
V/NativeLoader: Beginning load of avutil...
V/NativeLoader: avutil (null) was loaded normally!
V/NativeLoader: Beginning load of m...
V/NativeLoader: m (null) was loaded normally!
V/NativeLoader: Beginning load of dl...
V/NativeLoader: Loading the library normally failed: java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "__cxa_finalize" referenced by "/system/lib/libdl.so"...
at java.lang.Runtime.loadLibrary(Runtime.java:372)
at java.lang.System.loadLibrary(System.java:1076)
at com.getkeepsafe.relinker.SystemLibraryLoader.loadLibrary(SystemLibraryLoader.java:24)
at com.getkeepsafe.relinker.ReLinkerInstance.loadLibraryInternal(ReLinkerInstance.java:163)
at com.getkeepsafe.relinker.ReLinkerInstance.loadLibrary(ReLinkerInstance.java:136)
at com.getkeepsafe.relinker.ReLinkerInstance.loadLibrary(ReLinkerInstance.java:92)
at com.getkeepsafe.relinker.ReLinkerInstance.loadLibraryInternal(ReLinkerInstance.java:189)
at com.getkeepsafe.relinker.ReLinkerInstance.loadLibrary(ReLinkerInstance.java:136)
at com.getkeepsafe.relinker.ReLinkerInstance.loadLibrary(ReLinkerInstance.java:92)
at com.getkeepsafe.relinker.ReLinkerInstance.loadLibraryInternal(ReLinkerInstance.java:189)
at com.getkeepsafe.relinker.ReLinkerInstance.loadLibrary(ReLinkerInstance.java:136)
at com.getkeepsafe.relinker.ReLinkerInstance.loadLibrary(ReLinkerInstance.java:92)
at com.domain.domain.model.util.Utils$NativeLoader.Load(Utils.java:69)
at com.domain.domain.model.platform.youtube.PlatformYoutube.<init>(PlatformYoutube.java:35)
at com.domain.domain.model.platform.youtube.PlatformFactoryYoutube.createPlatform(PlatformFactoryYoutube.java:12)
at com.domain.domain.LogInActivity$1.doInBackground(LogInActivity.java:43)
at com.domain.domain.LogInActivity$1.doInBackground(LogInActivity.java:40)
at android.os.AsyncTask$2.call(AsyncTask.java:295)
at java.util.concurrent.FutureTask.run(FutureTask.java:237)
at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:234)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588)
at java.lang.Thread.run(Thread.java:818)
V/NativeLoader: dl (null) was not loaded normally, re-linking...
V/NativeLoader: Looking for lib/x86/libdl.so in APK...
(I'm using a manual installation with extraction of .so files into the folder)
DeKinci
on 1 Apr 2018
Also it happens only with libavcodec.so file, others are loading correctly
DeKinci
on 1 Apr 2018
I currently have no access to any Linux OS to compile sources. Could you please provide .so files?
DeKinci
on 2 Apr 2018
I've fixed the issue in the latest commit and the snapshots have been updated, it should work now. Make sure to call mvn with the -U option: http://bytedeco.org/builds/
saudet
on 2 Apr 2018
Thanks!
I'm using gradle, is it okay to specify group, name, version and set changing to true, or I need to do something more?
DeKinci
on 2 Apr 2018
You might want to clear the cache just to be sure...
saudet
on 2 Apr 2018
I've cleared the cache, deleted .gradle folder, cleaned and rebuilt the project, but
If I add javaCV like this:
implementation group: 'org.bytedeco.javacpp-presets', name: 'opencv-platform', version: '3.4.1-1.4.2-SNAPSHOT', changing: true
and try to sync, it failes with a bunch of failed to resolve errors:
If I add it in another way (which helped me with previous problems):
implementation (group: 'org.bytedeco.javacpp-presets', name: 'opencv-platform', version: '3.4.1-1.4.2-SNAPSHOT', changing: true) { exclude group: 'org.bytedeco.javacpp-presets' }
implementation group: 'org.bytedeco.javacpp-presets', name: 'ffmpeg', version: '3.4.1-1.4.2-SNAPSHOT', classifier: 'android-arm'
implementation group: 'org.bytedeco.javacpp-presets', name: 'ffmpeg', version: '3.4.1-1.4.2-SNAPSHOT', classifier: 'android-x86'
It failes with:
Error:Failed to resolve: org.bytedeco.javacpp-presets:ffmpeg:3.4.1-1.4.2-SNAPSHOT
If I try to update version to 3.4.2 (which Studio prompts):
implementation (group: 'org.bytedeco.javacpp-presets', name: 'opencv-platform', version: '3.4.1-1.4.2-SNAPSHOT', changing: true) { exclude group: 'org.bytedeco.javacpp-presets' }
implementation group: 'org.bytedeco.javacpp-presets', name: 'ffmpeg', version: '3.4.2-1.4.2-SNAPSHOT', classifier: 'android-arm'
implementation group: 'org.bytedeco.javacpp-presets', name: 'ffmpeg', version: '3.4.2-1.4.2-SNAPSHOT', classifier: 'android-x86'
It fails with:
Error:Failed to resolve: ffmpeg-android-arm
Error:Failed to resolve: ffmpeg-android-x86
And finally this way
implementation (group: 'org.bytedeco.javacpp-presets', name: 'opencv-platform', version: '3.4.1-1.4.2-SNAPSHOT', changing: true) { exclude group: 'org.bytedeco.javacpp-presets' }
implementation group: 'org.bytedeco.javacpp-presets', name: 'ffmpeg', version: '3.4.2-1.4.2-SNAPSHOT'
syncs fine, but on build failes with (doesn't matter if multiDexEnabled is set):
Error:Execution failed for task ':app:transformDexArchiveWithExternalLibsDexMergerForDebug`.
java.lang.RuntimeException: com.android.builder.dexing.DexArchiveMergerException: Unable to merge dex`
There is nothing new with those binaries. Like I said, I just fixed that
bug and that's it!
saudet
on 3 Apr 2018
I managed to overcome that (I accidentally copied opencv preset, not javacv) and now app crashes with fatal signal even if I don't use setpts.
Here is log:
detected 4 logical cores
[in @ 0xd921a600] Setting 'video_size' to value '1280x720'
[in @ 0xd921a600] Setting 'pix_fmt' to value '0'
[in @ 0xd921a600] Setting 'time_base' to value '1/30'
[in @ 0xd921a600] Setting 'pixel_aspect' to value '1/1'
[in @ 0xd921a600] w:1280 h:720 pixfmt:yuv420p tb:1/30 fr:0/1 sar:1/1 sws_param:
[setpts @ 0xd921a720] Setting 'expr' to value 'N'
[Parsed_movie_0 @ 0xd921b260] Setting 'filename' to value '/storage/emulated/0/AppName/img.png'
[NULL @ 0xdcea3000] Opening '/storage/emulated/0/AppName/img.png' for reading
[file @ 0xd9932460] Setting default whitelist 'file,crypto'
[png_pipe @ 0xdcea3000] Format png_pipe probed with size=2048 and score=99
[png_pipe @ 0xdcea3000] Before avformat_find_stream_info() pos: 0 bytes read:32768 seeks:0 nb_streams:1
[png_pipe @ 0xdcea3000] After avformat_find_stream_info() pos: 179559 bytes read:179559 seeks:0 frames:1
[Parsed_movie_0 @ 0xd921b260] seek_point:0 format_name:(null) file_name:/storage/emulated/0/AppName/img.png stream_index:-1
[auto_scaler_0 @ 0xd921b320] w:iw h:ih flags:'bilinear' interl:0
[Parsed_overlay_1 @ 0xd921b2c0] auto-inserting filter 'auto_scaler_0' between the filter 'Parsed_movie_0' and the filter 'Parsed_overlay_1'
[AVFilterGraph @ 0xdcea6240] query_formats: 5 queried, 3 merged, 1 already done, 0 delayed
[in @ 0xd921a600] TB:0.033333 FRAME_RATE:nan SAMPLE_RATE:nan
[auto_scaler_0 @ 0xd921b320] w:800 h:600 fmt:rgba sar:1/1 -> w:800 h:600 fmt:yuva420p sar:1/1 flags:0x2
[Parsed_overlay_1 @ 0xd921b2c0] main w:1280 h:720 fmt:yuv420p overlay w:800 h:600 fmt:yuva420p
[Parsed_overlay_1 @ 0xd921b2c0] [framesync @ 0xe0d293e0] Selected 1/150 time base
[Parsed_overlay_1 @ 0xd921b2c0] [framesync @ 0xe0d293e0] Sync level 2
[in @ 0xd921a600] Changing frame properties on the fly is not supported by all filters.
[in @ 0xd921a600] Changing frame properties on the fly is not supported by all filters.
[auto_scaler_0 @ 0xd921b320] w:800 h:600 fmt:rgba sar:0/1 -> w:800 h:600 fmt:yuva420p sar:0/1 flags:0x2
[Parsed_overlay_1 @ 0xd921b2c0] n:1.000000 t:0.000000 pos:nan x:0.000000 xi:0 y:0.000000 yi:0
A/libc: Fatal signal 11 (SIGSEGV), code 1, fault addr 0x0 in tid 5247 (pool-10-thread-)
And the filter:
FFmpegFrameFilter filter = new FFmpegFrameFilter("movie=" + path + "[img];[in][img]overlay", imageWidth, imageHeight);
[in @ 0xd921a600] Changing frame properties on the fly is not supported by all filters.
Don't do that!
saudet
on 3 Apr 2018
In any case this unit test is passing:
https://github.com/bytedeco/javacv/blob/master/platform/src/test/java/org/bytedeco/javacv/FrameFilterTest.java
Please modify it to make it fail in the way you describe and I will look into what to do to fix this. Thanks!
saudet
on 3 Apr 2018
Actually, I found what the issue was. Fixed with commit https://github.com/bytedeco/javacv/commit/8833f48b3d87da8d0845aa023cdef0b54669ab5c, so please try again! Thanks
saudet
on 3 Apr 2018
Well, maybe it is my fault because nothing changed. Still Fatal signal 11. What do you mean under do not change frame properties on the fly?
DeKinci
on 3 Apr 2018
The frames you are pushing do not match the pixel format, width, or height
that you set.
Please modify the unit test to make it fail, and I will fix it! Thanks
saudet
on 3 Apr 2018
So, is everything working well now?
Or are you unable to reproduce this issue outside of your application?
saudet
on 10 Apr 2018
Have not tried yet. Needed rest, but now I'm back
DeKinci
on 10 Apr 2018
I used this code (it's very similar to the test) on Android:
public class TestFFmpeg {
private static final String LOG_TAG = TestFFmpeg.class.getSimpleName();
public void run() {
try {
Log.d(LOG_TAG, "Setting hardware");
File video = FileHelper.getFileFromAppDir("video.mp4");
String overlayPath = FileHelper.getFileFromAppDir("img.png").getAbsolutePath();
String output = FileHelper.createFileInAppDir("test" + System.currentTimeMillis() / 1000 + ".mp4").getAbsolutePath();
FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(video);
grabber.setPixelFormat(AV_PIX_FMT_YUV420P); //FIXME <--- bad one
FFmpegFrameFilter filter = new FFmpegFrameFilter("movie=" + overlayPath + "[img];[in][img]overlay", 1280, 720);
filter.setPixelFormat(grabber.getPixelFormat());
FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(output, 1280, 720, 2);
recorder.setPixelFormat(filter.getPixelFormat());
setUpRecorder(recorder);
Log.d(LOG_TAG, "Starting recording");
recorder.start();
filter.start();
grabber.start();
long startTime = System.currentTimeMillis();
Frame grabbedFrame;
Frame filteredFrame;
Log.d(LOG_TAG, "Recording started");
while ((grabbedFrame = grabber.grab()) != null) {
Log.v(LOG_TAG, "New grabbed frame");
filter.push(grabbedFrame);
while ((filteredFrame = filter.pull()) != null) {
Log.v(LOG_TAG, "New filtered frame");
long t = 1000 * (System.currentTimeMillis() - startTime);
if (t > recorder.getTimestamp())
recorder.setTimestamp(t);
recorder.record(filteredFrame);
}
}
int elapsedSeconds = (int) (startTime - System.currentTimeMillis() / 1000);
Log.d(LOG_TAG, "Finishing recording in " + elapsedSeconds / 60 + "m " + elapsedSeconds % 60 + "s");
grabber.stop();
filter.stop();
recorder.stop();
Log.d(LOG_TAG, "Recording finished");
} catch (Exception e) {
e.printStackTrace();
}
}
private void setUpRecorder(FFmpegFrameRecorder recorder) {
recorder.setInterleaved(true);
recorder.setFormat("mp4");
recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);
recorder.setFrameRate(30);
recorder.setGopSize(30 * 2);
recorder.setVideoBitrate(3_000_000);
recorder.setVideoOption("tune", "zerolatency");
recorder.setVideoOption("preset", "ultrafast");
recorder.setVideoOption("crf", "28");
recorder.setAudioCodec(avcodec.AV_CODEC_ID_AAC);
recorder.setSampleRate(44_100);
recorder.setAudioQuality(0);
recorder.setAudioChannels(2);
recorder.setAudioBitrate(128_000);
recorder.setAudioOption("crf", "0");
}
}
And here is what I got:
If I do not call .setPixelFormat() anywhere, everything works fine, but the imaged is tripled by width (not rotated) and black-and-white.
If I do not call .setPixelFormat() on grabber (has a FIXME comment), but do on filter and recorder I get exception:
W/System.err: Error: [libx264 @ 0xe0fee000] Specified pixel format bgr24 is invalid or not supported
W/System.err: org.bytedeco.javacv.FrameRecorder$Exception: avcodec_open2() error -22: Could not open video codec.
If I set pixel format on grabber to AV_PIX_FMT_YUV420P I get this:
I/System.out: [libx264 @ 0xe11ed800] using cpu capabilities: none!
[libx264 @ 0xe11ed800] using cpu capabilities: none!
I/System.out: [libx264 @ 0xe11ed800] profile Constrained Baseline, level 3.1
[libx264 @ 0xe11ed800] profile Constrained Baseline, level 3.1
[libx264 @ 0xe11ed800] 264 - core 152 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=4 lookahead_threads=4 sliced_threads=1 slices=4 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=60 keyint_min=6 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=28.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
[libx264 @ 0xe11ed800] 264 - core 152 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=4 lookahead_threads=4 sliced_threads=1 slices=4 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=60 keyint_min=6 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=28.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
I/System.out: [file @ 0xe10aa5e0] Setting default whitelist 'file,crypto'
Output #0, mp4, to '/storage/emulated/0/appname/test1523455393.mp4':
I/System.out: Output #0, mp4, to '/storage/emulated/0/appname/test1523455393.mp4':
Metadata:
Metadata:
encoder :
encoder :
Lavf57.83.100
Lavf57.83.100
Stream #0:0
Stream #0:0
, 0, 1/15360
: Video: h264 (Constrained Baseline), 1 reference frame (avc1 / 0x31637661), yuv420p, 1280x720 (0x0), 0/1, q=2-31, 3000 kb/s
: Video: h264 (Constrained Baseline), 1 reference frame (avc1 / 0x31637661), yuv420p, 1280x720 (0x0), 0/1, q=2-31, 3000 kb/s
,
,
15360 tbn
15360 tbn
Stream #0:1
Stream #0:1
, 0, 1/44100
: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, delay 1024, 128 kb/s
: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, delay 1024, 128 kb/s
I/System.out: detected 4 logical cores
[in @ 0xe10a06e0] Setting 'video_size' to value '1280x720'
[in @ 0xe10a06e0] Setting 'pix_fmt' to value '0'
[in @ 0xe10a06e0] Setting 'time_base' to value '1/30'
[in @ 0xe10a06e0] Setting 'pixel_aspect' to value '1/1'
[in @ 0xe10a06e0] w:1280 h:720 pixfmt:yuv420p tb:1/30 fr:0/1 sar:1/1 sws_param:
I/System.out: [setpts @ 0xe10a0980] Setting 'expr' to value 'N'
I/System.out: [Parsed_movie_0 @ 0xe10a0b00] Setting 'filename' to value '/storage/emulated/0/appname/img.png'
[NULL @ 0xe0dfee00] Opening '/storage/emulated/0/appname/img.png' for reading
[file @ 0xe10a9b60] Setting default whitelist 'file,crypto'
I/System.out: [png_pipe @ 0xe0dfee00] Format png_pipe probed with size=2048 and score=99
[png_pipe @ 0xe0dfee00] Before avformat_find_stream_info() pos: 0 bytes read:32768 seeks:0 nb_streams:1
I/System.out: [png_pipe @ 0xe0dfee00] After avformat_find_stream_info() pos: 179559 bytes read:179559 seeks:0 frames:1
[Parsed_movie_0 @ 0xe10a0b00] seek_point:0 format_name:(null) file_name:/storage/emulated/0/appname/img.png stream_index:-1
[auto_scaler_0 @ 0xe10a0bc0] w:iw h:ih flags:'bilinear' interl:0
[Parsed_overlay_1 @ 0xe10a0b60] auto-inserting filter 'auto_scaler_0' between the filter 'Parsed_movie_0' and the filter 'Parsed_overlay_1'
[AVFilterGraph @ 0xe127f940] query_formats: 5 queried, 3 merged, 1 already done, 0 delayed
[in @ 0xe10a06e0] TB:0.033333 FRAME_RATE:nan SAMPLE_RATE:nan
I/System.out: [auto_scaler_0 @ 0xe10a0bc0] w:800 h:600 fmt:rgba sar:1/1 -> w:800 h:600 fmt:yuva420p sar:1/1 flags:0x2
[Parsed_overlay_1 @ 0xe10a0b60] main w:1280 h:720 fmt:yuv420p overlay w:800 h:600 fmt:yuva420p
[Parsed_overlay_1 @ 0xe10a0b60] [framesync @ 0xe85f1ca0] Selected 1/150 time base
[Parsed_overlay_1 @ 0xe10a0b60] [framesync @ 0xe85f1ca0] Sync level 2
I/System.out: [NULL @ 0xe0dffa00] Opening '/storage/emulated/0/appname/video.mp4' for reading
[file @ 0xe10aa0a0] Setting default whitelist 'file,crypto'
I/System.out: [mov,mp4,m4a,3gp,3g2,mj2 @ 0xe0dffa00] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0xe0dffa00] ISO: File Type Major Brand: isom
I/System.out: [mov,mp4,m4a,3gp,3g2,mj2 @ 0xe0dffa00] Unknown dref type 0x206c7275 size 12
[mov,mp4,m4a,3gp,3g2,mj2 @ 0xe0dffa00] Processing st: 0, edit list 0 - media time: 0, duration: 2186752
I/System.out: [mov,mp4,m4a,3gp,3g2,mj2 @ 0xe0dffa00] Unknown dref type 0x206c7275 size 12
I/System.out: [mov,mp4,m4a,3gp,3g2,mj2 @ 0xe0dffa00] Processing st: 1, edit list 0 - media time: 0, duration: 8201232
[mov,mp4,m4a,3gp,3g2,mj2 @ 0xe0dffa00] Before avformat_find_stream_info() pos: 31491130 bytes read:130297 seeks:1 nb_streams:2
[h264 @ 0xe9be1c00] nal_unit_type: 7, nal_ref_idc: 3
[h264 @ 0xe9be1c00] nal_unit_type: 8, nal_ref_idc: 3
I/System.out: [h264 @ 0xe9be1c00] nal_unit_type: 5, nal_ref_idc: 3
[h264 @ 0xe9be1c00] Reinit context to 1280x720, pix_fmt: yuv420p
I/System.out: [mov,mp4,m4a,3gp,3g2,mj2 @ 0xe0dffa00] All info found
I/System.out: [mov,mp4,m4a,3gp,3g2,mj2 @ 0xe0dffa00] After avformat_find_stream_info() pos: 106237 bytes read:236486 seeks:2 frames:2
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/storage/emulated/0/appname/video.mp4':
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/storage/emulated/0/appname/video.mp4':
Metadata:
Metadata:
major_brand :
major_brand :
isom
isom
minor_version :
minor_version :
512
512
compatible_brands:
compatible_brands:
isomiso2avc1mp41
isomiso2avc1mp41
creation_time :
creation_time :
1970-01-01T00:00:00.000000Z
1970-01-01T00:00:00.000000Z
encoder :
encoder :
Lavf53.24.2
Lavf53.24.2
I/System.out: Duration:
Duration:
00:02:50.86
00:02:50.86
, start:
, start:
0.000000
0.000000
, bitrate:
, bitrate:
1474 kb/s
1474 kb/s
Stream #0:0
Stream #0:0
(und)
(und)
, 1, 1/12800
: Video: h264 (Main), 1 reference frame (avc1 / 0x31637661), yuv420p(left), 1280x720 [SAR 1:1 DAR 16:9], 0/1, 1086 kb/s
: Video: h264 (Main), 1 reference frame (avc1 / 0x31637661), yuv420p(left), 1280x720 [SAR 1:1 DAR 16:9], 0/1, 1086 kb/s
I/System.out: ,
,
25 fps,
25 fps,
25 tbr,
25 tbr,
12800 tbn,
12800 tbn,
50 tbc
50 tbc
(default)
(default)
Metadata:
Metadata:
creation_time :
creation_time :
1970-01-01T00:00:00.000000Z
1970-01-01T00:00:00.000000Z
I/System.out: handler_name :
handler_name :
VideoHandler
VideoHandler
Stream #0:1
Stream #0:1
(und)
(und)
, 1, 1/48000
: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 5.1, fltp, 383 kb/s
: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 5.1, fltp, 383 kb/s
(default)
(default)
I/System.out: Metadata:
Metadata:
creation_time :
creation_time :
1970-01-01T00:00:00.000000Z
1970-01-01T00:00:00.000000Z
handler_name :
handler_name :
SoundHandler
SoundHandler
[h264 @ 0xe12bf000] nal_unit_type: 7, nal_ref_idc: 3
[h264 @ 0xe12bf000] nal_unit_type: 8, nal_ref_idc: 3
D/TestFFmpeg: Recording started
I/System.out: [SWR @ 0xd9382000] Using fltp internally between filters
V/TestFFmpeg: New grabbed frame
I/System.out: [h264 @ 0xe12bf000] nal_unit_type: 5, nal_ref_idc: 3
[h264 @ 0xe12bf000] Reinit context to 1280x720, pix_fmt: yuv420p
V/TestFFmpeg: New grabbed frame
I/System.out: [in @ 0xe10a06e0] Changing frame properties on the fly is not supported by all filters.
[in @ 0xe10a06e0] Changing frame properties on the fly is not supported by all filters.
I/System.out: [auto_scaler_0 @ 0xe10a0bc0] w:800 h:600 fmt:rgba sar:0/1 -> w:800 h:600 fmt:yuva420p sar:0/1 flags:0x2
I/System.out: [Parsed_overlay_1 @ 0xe10a0b60] n:1.000000 t:0.000000 pos:nan x:0.000000 xi:0 y:0.000000 yi:0
A/libc: Fatal signal 11 (SIGSEGV), code 1, fault addr 0x0 in tid 10506 (nci.appname)
And it looks suspiciously similar to what I get while streaming from camera! (Attention to the last line).
Also it works perfectly fine without using the filter too!
Worth noting, that video is not in YUV420P format, but it is ok for grabber->recorder usage (video becomes grayscale).
It seems to me that problem is in using filter with YUV420 images
DeKinci
on 11 Apr 2018
Finally after some format filter expirements this works just fine:
public class TestFFmpeg {
private static final String LOG_TAG = TestFFmpeg.class.getSimpleName();
public void run() {
try {
Log.d(LOG_TAG, "Setting hardware");
File video = FileHelper.getFileFromAppDir("video.mp4");
String fontPath = FileHelper.getFileFromAppDir("font.ttf").getAbsolutePath();
String overlayPath = FileHelper.getFileFromAppDir("img.png").getAbsolutePath();
String output = FileHelper.createFileInAppDir("test" + System.currentTimeMillis() / 1000 + ".mp4").getAbsolutePath();
String filterString = "movie=" + overlayPath + "[img];" +
"[in][img]overlay,format=pix_fmts=bgr24," +
"drawtext=fontfile=" + fontPath + ":text='TEST TEST TEST TEST'";
FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(video);
FFmpegFrameFilter filter = new FFmpegFrameFilter(filterString, 1280, 720);
FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(output, 1280, 720, 2);
setUpRecorder(recorder);
Log.d(LOG_TAG, "Starting recording");
recorder.start();
filter.start();
grabber.start();
long startTime = System.currentTimeMillis();
Frame grabbedFrame;
Frame filteredFrame;
Log.d(LOG_TAG, "Recording started");
while ((grabbedFrame = grabber.grab()) != null) {
Log.v(LOG_TAG, "New grabbed frame");
filter.push(grabbedFrame);
while ((filteredFrame = filter.pull()) != null) {
Log.v(LOG_TAG, "New filtered frame");
long t = 1000 * (System.currentTimeMillis() - startTime);
if (t > recorder.getTimestamp())
recorder.setTimestamp(t);
recorder.record(filteredFrame);
}
}
int elapsedSeconds = (int) (System.currentTimeMillis() - startTime) / 1000;
Log.d(LOG_TAG, "Finishing recording in " + elapsedSeconds / 60 + "m " + elapsedSeconds % 60 + "s");
grabber.stop();
filter.stop();
recorder.stop();
Log.d(LOG_TAG, "Recording finished");
} catch (Exception e) {
e.printStackTrace();
}
}
private void setUpRecorder(FFmpegFrameRecorder recorder) {
recorder.setInterleaved(true);
recorder.setFormat("mp4");
recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);
recorder.setFrameRate(30);
recorder.setGopSize(30 * 2);
recorder.setVideoBitrate(3_000_000);
recorder.setVideoOption("tune", "zerolatency");
recorder.setVideoOption("preset", "ultrafast");
recorder.setVideoOption("crf", "28");
recorder.setAudioCodec(avcodec.AV_CODEC_ID_AAC);
recorder.setSampleRate(44_100);
recorder.setAudioQuality(0);
recorder.setAudioChannels(2);
recorder.setAudioBitrate(128_000);
recorder.setAudioOption("crf", "0");
}
}
Thanks for help!
DeKinci
on 11 Apr 2018
Great! Thanks for the feedback. What you have looks good.
BTW, by default FFmpeg filters can output in any format so we either have to force the output to something like bgr24 like you did, or call record() with the pixel format, see issue #887.
saudet
on 11 Apr 2018
It does not work with streaming. When I call record with pixel format too. Just change output to rtmp url and it fales with
Fatal signal 11 (SIGSEGV), code 1, fault addr 0x0 in tid 4409 (nci.appname)
DeKinci
on 12 Apr 2018
Does the same thing happen with Java SE? Or does it happen only with Android?
saudet
on 12 Apr 2018
Does not works for streaming (fine for saving).
Code:
public class Test {
public static void main(String[] args) {
new Test().run();
}
private void run() {
try {
System.out.println("Setting hardware");
File video = new File("stream.mp4");
String fontPath = new File("font.ttf").getPath();
String overlayPath = new File("img.png").getPath();
File outputFile = new File("test" + System.currentTimeMillis() / 1000 + ".mp4");
outputFile.createNewFile();
String output = outputFile.getAbsolutePath();
output = "rtmp://a.rtmp.youtube.com/live2/980m-48ud-j2de-8j41";
String filterString = "movie=" + overlayPath + "[img];" +
"[in][img]overlay,format=pix_fmts=bgr24," +
"drawtext=fontfile=" + fontPath + ":text='TEST TEST TEST TEST'";
FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(video);
FFmpegFrameFilter filter = new FFmpegFrameFilter(filterString, 1280, 720);
FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(output, 1280, 720, 2);
setUpRecorder(recorder);
System.out.println("Starting recording");
recorder.start();
filter.start();
grabber.start();
long startTime = System.currentTimeMillis();
Frame grabbedFrame;
Frame filteredFrame;
System.out.println("Recording started");
while ((grabbedFrame = grabber.grab()) != null) {
System.out.println("New grabbed frame");
filter.push(grabbedFrame);
while ((filteredFrame = filter.pull()) != null) {
System.out.println("New filtered frame");
long t = 1000 * (System.currentTimeMillis() - startTime);
if (t > recorder.getTimestamp())
recorder.setTimestamp(t);
recorder.record(filteredFrame);
}
}
int elapsedSeconds = (int) (System.currentTimeMillis() - startTime) / 1000;
System.out.println("Finishing recording in " + elapsedSeconds / 60 + "m " + elapsedSeconds % 60 + "s");
grabber.stop();
filter.stop();
recorder.stop();
System.out.println("Recording finished");
} catch (Exception e) {
e.printStackTrace();
}
}
private void setUpRecorder(FFmpegFrameRecorder recorder) {
recorder.setInterleaved(true);
recorder.setFormat("mp4");
recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);
recorder.setFrameRate(30);
recorder.setGopSize(30 * 2);
recorder.setVideoBitrate(3_000_000);
recorder.setVideoOption("tune", "zerolatency");
recorder.setVideoOption("preset", "ultrafast");
recorder.setVideoOption("crf", "28");
recorder.setAudioCodec(avcodec.AV_CODEC_ID_AAC);
recorder.setSampleRate(44_100);
recorder.setAudioQuality(0);
recorder.setAudioChannels(2);
recorder.setAudioBitrate(128_000);
recorder.setAudioOption("crf", "0");
}
}
Log:
Setting hardware
Starting recording
Recording started
New grabbed frame
New grabbed frame
New filtered frame
#
# A fatal error has been detected by the Java Runtime Environment:
#
# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00007fff0fe48c19, pid=11420, tid=6532
#
# JRE version: Java(TM) SE Runtime Environment (10.0+46) (build 10+46)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (10+46, mixed mode, tiered, compressed oops, g1 gc, windows-amd64)
# Problematic frame:
# C [avformat-57.dll+0xd8c19]
#
# No core dump will be written. Minidumps are not enabled by default on client versions of Windows
#
# An error report file with more information is saved as:
# E:\Users\knd20\IdeaProjects\javacvtest\hs_err_pid11420.log
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
[libx264 @ 00000243a52ae260] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 00000243a52ae260] profile Constrained Baseline, level 3.1
[libx264 @ 00000243a52ae260] 264 - core 152 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=8 lookahead_threads=8 sliced_threads=1 slices=8 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=60 keyint_min=6 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=28.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
[mp4 @ 00000243a52acb80] muxer does not support non seekable output
Output #0, mp4, to 'rtmp://a.rtmp.youtube.com/live2/980m-48ud-j2de-8j41':
Metadata:
encoder : Lavf57.83.100
Stream #0:0: Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 1280x720, q=2-31, 3000 kb/s, 30 tbn
Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'E:\Users\knd20\IdeaProjects\comdekinci\stream.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
creation_time : 1970-01-01T00:00:00.000000Z
encoder : Lavf53.24.2
Duration: 00:02:50.86, start: 0.000000, bitrate: 1474 kb/s
Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 1086 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
Metadata:
creation_time : 1970-01-01T00:00:00.000000Z
handler_name : VideoHandler
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 5.1, fltp, 383 kb/s (default)
Metadata:
creation_time : 1970-01-01T00:00:00.000000Z
handler_name : SoundHandler
Process finished with exit code 1
Java log:
hs_err_pid11420.log
DeKinci
on 13 Apr 2018
Looks like ffmpeg problem, is there any way to fix it urgently (mb some temporary solution)? Also I wonder why it worked previously (on Android)
DeKinci
on 13 Apr 2018
You could try to revert to an old version of FFmpeg that is known to work
like 3.4.1. Was that working?
saudet
on 13 Apr 2018
How to revert to older FFmpeg version? I tried javacv 1.4.1 and it's all the same
DeKinci
on 14 Apr 2018
Try with an even older version JavaCV. What's the last version that isn't
crashing?
saudet
on 14 Apr 2018
Maybe I am doing something wrong? 1.4 failes too.
On 1.3.3 got an exception for drawtext filter:
org.bytedeco.javacv.FrameFilter$Exception: avfilter_graph_parse_ptr()
at org.bytedeco.javacv.FFmpegFrameFilter.startUnsafe(FFmpegFrameFilter.java:233)
at org.bytedeco.javacv.FFmpegFrameFilter.start(FFmpegFrameFilter.java:155)
at Test.run(Test.java:37)
at Test.main(Test.java:11)
[libx264 @ 000002c3291ce0c0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 AVX2 LZCNT BMI2
[libx264 @ 000002c3291ce0c0] profile Constrained Baseline, level 3.1
[libx264 @ 000002c3291ce0c0] 264 - core 148 - H.264/MPEG-4 AVC codec - Copyleft 2003-2016 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=8 lookahead_threads=8 sliced_threads=1 slices=8 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=60 keyint_min=6 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=28.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
[mp4 @ 000002c3291cca40] muxer does not support non seekable output
Output #0, mp4, to 'rtmp://a.rtmp.youtube.com/live2/980m-48ud-j2de-8j41':
Metadata:
encoder : Lavf57.56.100
Stream #0:0: Video: h264 (Constrained Baseline) ([33][0][0][0] / 0x0021), yuv420p, 1280x720, q=2-31, 3000 kb/s, 30 tbn
Stream #0:1: Audio: aac (LC) ([64][0][0][0] / 0x0040), 44100 Hz, stereo, fltp, 128 kb/s
No such filter: 'drawtext'
Removed it and same error
1.3 same error
Cannot resolve under 1.3
DeKinci
on 14 Apr 2018
drawtext wasn't supported previously. If you remove drawtext, does it not
crash with 1.3.3?
saudet
on 14 Apr 2018
I removed it and got crash on everything including 1.3
DeKinci
on 14 Apr 2018
Seems to be an issue with FFmpeg then. Does it do the same with the ffmpeg
command line program? In any case, set the log level to debug again, and
let's check for anything suspicious in the message it prints to the console.
saudet
on 14 Apr 2018
I will use 1.4.2-SNAPSHOT now on. Also I removed drawtext.
[libx264 @ 000001680669e2c0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 000001680669e2c0] profile Constrained Baseline, level 3.1
[libx264 @ 000001680669e2c0] 264 - core 152 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=8 lookahead_threads=8 sliced_threads=1 slices=8 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=60 keyint_min=6 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=28.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
[rtmp @ 000001682cfa6ee0] No default whitelist set
[tcp @ 000001682d083f80] No default whitelist set
[rtmp @ 000001682cfa6ee0] Handshaking...
[rtmp @ 000001682cfa6ee0] Type answer 3
[rtmp @ 000001682cfa6ee0] Server version 4.0.0.1
[rtmp @ 000001682cfa6ee0] Proto = rtmp, path = /live2/980m-48ud-j2de-8j41, app = live2, fname = 980m-48ud-j2de-8j41
[rtmp @ 000001682cfa6ee0] Window acknowledgement size = 2500000
[rtmp @ 000001682cfa6ee0] Max sent, unacked = 10000000
[rtmp @ 000001682cfa6ee0] New incoming chunk size = 256
[rtmp @ 000001682cfa6ee0] Releasing stream...
[rtmp @ 000001682cfa6ee0] FCPublish stream...
[rtmp @ 000001682cfa6ee0] Creating stream...
[rtmp @ 000001682cfa6ee0] Sending publish command for '980m-48ud-j2de-8j41'
[mp4 @ 000001680669cbe0] muxer does not support non seekable output
Output #0, mp4, to 'rtmp://a.rtmp.youtube.com/live2/980m-48ud-j2de-8j41':
Metadata:
encoder : Lavf57.83.100
Stream #0:0, 0, 1/30: Video: h264 (Constrained Baseline), 1 reference frame (avc1 / 0x31637661), yuv420p, 1280x720 (0x0), 0/1, q=2-31, 3000 kb/s, 30 tbn
Stream #0:1, 0, 1/44100: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, delay 1024, 128 kb/s
detected 8 logical cores
[in @ 000001682d08ecc0] Setting 'video_size' to value '1280x720'
[in @ 000001682d08ecc0] Setting 'pix_fmt' to value '3'
[in @ 000001682d08ecc0] Setting 'time_base' to value '1/30'
[in @ 000001682d08ecc0] Setting 'pixel_aspect' to value '1/1'
[in @ 000001682d08ecc0] w:1280 h:720 pixfmt:bgr24 tb:1/30 fr:0/1 sar:1/1 sws_param:
[setpts @ 000001682d08efe0] Setting 'expr' to value 'N'
[Parsed_movie_0 @ 000001682d08f740] Setting 'filename' to value 'img.png'
[NULL @ 000001682d08f960] Opening 'img.png' for reading
[file @ 000001682d090020] Setting default whitelist 'file,crypto'
[png_pipe @ 000001682d08f960] Format png_pipe probed with size=2048 and score=99
[png_pipe @ 000001682d08f960] Before avformat_find_stream_info() pos: 0 bytes read:6696 seeks:0 nb_streams:1
[png_pipe @ 000001682d08f960] After avformat_find_stream_info() pos: 6696 bytes read:6696 seeks:0 frames:1
[Parsed_movie_0 @ 000001682d08f740] seek_point:0 format_name:(null) file_name:img.png stream_index:-1
[Parsed_format_2 @ 000001682d097500] compat: called with args=[pix_fmts=bgr24]
[Parsed_format_2 @ 000001682d097500] Setting 'pix_fmts' to value 'bgr24'
[auto_scaler_0 @ 000001682d093440] w:iw h:ih flags:'bilinear' interl:0
[Parsed_overlay_1 @ 000001682d095dc0] auto-inserting filter 'auto_scaler_0' between the filter 'setpts' and the filter 'Parsed_overlay_1'
[auto_scaler_1 @ 000001682d092ee0] w:iw h:ih flags:'bilinear' interl:0
[Parsed_overlay_1 @ 000001682d095dc0] auto-inserting filter 'auto_scaler_1' between the filter 'Parsed_movie_0' and the filter 'Parsed_overlay_1'
[auto_scaler_2 @ 000001682d097f00] w:iw h:ih flags:'bilinear' interl:0
[Parsed_format_2 @ 000001682d097500] auto-inserting filter 'auto_scaler_2' between the filter 'Parsed_overlay_1' and the filter 'Parsed_format_2'
[AVFilterGraph @ 000001682d085b00] query_formats: 6 queried, 2 merged, 3 already done, 0 delayed
[in @ 000001682d08ecc0] TB:0.033333 FRAME_RATE:nan SAMPLE_RATE:nan
[auto_scaler_0 @ 000001682d093440] w:1280 h:720 fmt:bgr24 sar:1/1 -> w:1280 h:720 fmt:yuva420p sar:1/1 flags:0x2
[auto_scaler_1 @ 000001682d092ee0] w:829 h:101 fmt:rgba sar:1/1 ->Recording started
New grabbed frame
New grabbed frame
New filtered frame
#
# A fatal error has been detected by the Java Runtime Environment:
#
# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00007fff0ef48c19, pid=13164, tid=3296
#
# JRE version: Java(TM) SE Runtime Environment (10.0+46) (build 10+46)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (10+46, mixed mode, tiered, compressed oops, g1 gc, windows-amd64)
# Problematic frame:
# C [avformat-57.dll+0xd8c19]
#
# No core dump will be written. Minidumps are not enabled by default on client versions of Windows
#
# An error report file with more information is saved as:
# E:\Users\knd20\IdeaProjects\comdekinci\hs_err_pid13164.log
w:829 h:101 fmt:yuva420p sar:1/1 flags:0x2
[Parsed_overlay_1 @ 000001682d095dc0] main w:1280 h:720 fmt:yuva420p overlay w:829 h:101 fmt:yuva420p
[Parsed_overlay_1 @ 000001682d095dc0] [framesync @ 000001682d095ec8] Selected 1/150 time base
[Parsed_overlay_1 @ 000001682d095dc0] [framesync @ 000001682d095ec8] Sync level 2
[auto_scaler_2 @ 000001682d097f00] w:1280 h:720 fmt:yuva420p sar:1/1 -> w:1280 h:720 fmt:bgr24 sar:1/1 flags:0x2
[NULL @ 000001682d112260] Opening 'E:\Users\knd20\IdeaProjects\comdekinci\stream.mp4' for reading
[file @ 000001682d1128a0] Setting default whitelist 'file,crypto'
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001682d112260] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001682d112260] ISO: File Type Major Brand: isom
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001682d112260] Unknown dref type 0x206c7275 size 12
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001682d112260] Processing st: 0, edit list 0 - media time: 0, duration: 2186752
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001682d112260] Unknown dref type 0x206c7275 size 12
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001682d112260] Processing st: 1, edit list 0 - media time: 0, duration: 8201232
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001682d112260] Before avformat_find_stream_info() pos: 31491130 bytes read:130297 seeks:1 nb_streams:2
[h264 @ 000001682d126b80] nal_unit_type: 7, nal_ref_idc: 3
[h264 @ 000001682d126b80] nal_unit_type: 8, nal_ref_idc: 3
[h264 @ 000001682d126b80] nal_unit_type: 5, nal_ref_idc: 3
[h264 @ 000001682d126b80] Reinit context to 1280x720, pix_fmt: yuv420p
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001682d112260] All info found
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001682d112260] After avformat_find_stream_info() pos: 106237 bytes read:236486 seeks:2 frames:2
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'E:\Users\knd20\IdeaProjects\comdekinci\stream.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
creation_time : 1970-01-01T00:00:00.000000Z
encoder : Lavf53.24.2
Duration: 00:02:50.86, start: 0.000000, bitrate: 1474 kb/s
Stream #0:0(und), 1, 1/12800: Video: h264 (Main), 1 reference frame (avc1 / 0x31637661), yuv420p(left), 1280x720 [SAR 1:1 DAR 16:9], 0/1, 1086 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
Metadata:
creation_time : 1970-01-01T00:00:00.000000Z
handler_name : VideoHandler
Stream #0:1(und), 1, 1/48000: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 5.1, fltp, 383 kb/s (default)
Metadata:
creation_time : 1970-01-01T00:00:00.000000Z
handler_name : SoundHandler
[h264 @ 000001682eac00a0] nal_unit_type: 7, nal_ref_idc: 3
[h264 @ 000001682eac00a0] nal_unit_type: 8, nal_ref_idc: 3
[SWR @ 000001682eb46dc0] Using fltp internally between filters
[h264 @ 000001682eac00a0] nal_unit_type: 5, nal_ref_idc: 3
[h264 @ 000001682eac00a0] Reinit context to 1280x720, pix_fmt: yuv420p
[auto_scaler_0 @ 000001682d093440] w:1280 h:720 fmt:bgr24 sar:0/1 -> w:1280 h:720 fmt:yuva420p sar:0/1 flags:0x2
[auto_scaler_1 @ 000001682d092ee0] w:829 h:101 fmt:rgba sar:0/1 -> w:829 h:101 fmt:yuva420p sar:0/1 flags:0x2
[Parsed_overlay_1 @ 000001682d095dc0] n:1.000000 t:0.000000 pos:nan x:0.000000 xi:0 y:0.000000 yi:0
[auto_scaler_2 @ 000001682d097f00] w:1280 h:720 fmt:yuva420p sar:0/1 -> w:1280 h:720 fmt:bgr24 sar:0/1 flags:0x2
[libx264 @ 000001680669e2c0] frame= 0 QP=25.00 NAL=3 Slice:I Poc:0 I:3600 P:0 SKIP:0 size=115606 bytes
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
Thanks! Could you also try without FFmpegFrameFilter entirely?
saudet
on 14 Apr 2018
Here it is
[libx264 @ 000001b7d700e2c0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 000001b7d700e2c0] profile Constrained Baseline, level 3.1
[libx264 @ 000001b7d700e2c0] 264 - core 152 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=8 lookahead_threads=8 sliced_threads=1 slices=8 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=60 keyint_min=6 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=28.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
[rtmp @ 000001b7fd916ee0] No default whitelist set
[tcp @ 000001b7fd9f3f80] No default whitelist set
[rtmp @ 000001b7fd916ee0] Handshaking...
[rtmp @ 000001b7fd916ee0] Type answer 3
[rtmp @ 000001b7fd916ee0] Server version 4.0.0.1
[rtmp @ 000001b7fd916ee0] Proto = rtmp, path = /live2/980m-48ud-j2de-8j41, app = live2, fname = 980m-48ud-j2de-8j41
[rtmp @ 000001b7fd916ee0] Window acknowledgement size = 2500000
[rtmp @ 000001b7fd916ee0] Max sent, unacked = 10000000
[rtmp @ 000001b7fd916ee0] New incoming chunk size = 256
[rtmp @ 000001b7fd916ee0] Releasing stream...
[rtmp @ 000001b7fd916ee0] FCPublish stream...
[rtmp @ 000001b7fd916ee0] Creating stream...
[rtmp @ 000001b7fd916ee0] Sending publish command for '980m-48ud-j2de-8j41'
[mp4 @ 000001b7d700cbe0] muxer does not support non seekable output
Output #0, mp4, to 'rtmp://a.rtmp.youtube.com/live2/980m-48ud-j2de-8j41':
Metadata:
encoder : Lavf57.83.100
Stream #0:0, 0, 1/30: Video: h264 (Constrained Baseline), 1 reference frame (avc1 / 0x31637661), yuv420p, 1280x720 (0x0), 0/1, q=2-31, 3000 kb/s, 30 tbn
Stream #0:1, 0, 1/44100: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, delay 1024, 128 kb/s
[NULL @ 000001b7fd9f5500] Opening 'E:\Users\knd20\IdeaProjects\comdekinci\stream.mp4' for reading
[file @ 000001b7fd9f5bc0] Setting default whitelist 'file,crypto'
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001b7fd9f5500] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001b7fd9f5500] ISO: File Type Major Brand: isom
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001b7fd9f5500] Unknown dref type 0x206c7275 size 12
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001b7fd9f5500] Processing st: 0, edit list 0 - media time: 0, duration: 2186752
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001b7fd9f5500] Unknown dref type 0x206c7275 size 12
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001b7fd9f5500] Processing st: 1, edit list 0 - media time: 0, duration: 8201232
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001b7fd9f5500] Before avformat_find_stream_info() pos: 31491130 bytes read:130297 seeks:1 nb_streams:2
[h264 @ 000001b7fda0fce0] nal_unit_type: 7, nal_ref_idc: 3
[h264 @ 000001b7fda0fce0] nal_unit_type: 8, nal_ref_idc: 3
[h264 @ 000001b7fda0fce0] nal_unit_type: 5, nal_ref_idc: 3
[h264 @ 000001b7fda0fce0] Reinit context to 1280x720, pix_fmt: yuv420p
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001b7fd9f5500] All info found
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001b7fd9f5500] After avformat_find_stream_info() pos: 106237 bytes read:236486 seeks:2 frames:2
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'E:\Users\knd20\IdeaProjects\comdekinci\stream.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
creation_time : 1970-01-01T00:00:00.000000Z
encoder : Lavf53.24.2
Duration: 00:02:50.86, start: 0.000000, bitrate: 1474 kb/s
Stream #0:0(und), 1, 1/12800: Video: h264 (Main), 1 reference frame (avc1 / 0x31637661), yuv420p(left), 1280x720 [SAR 1:1 DAR 16:9], 0/1, 1086 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
Metadata:
creation_time : 1970-01-01T00:00:00.000000Z
handler_name : VideoHandler
Stream #0:1(und), 1, 1/48000: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 5.1, fltp, 383 kb/s (default)
Metadata:
creation_time : 1970-01-01T00:00:00.000000Z
handler_nRecording started
New grabbed frame
New grabbed frame
#
# A fatal error has been detected by the Java Runtime Environment:
#
# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00007fff0a238c19, pid=13448, tid=12952
#
# JRE version: Java(TM) SE Runtime Environment (10.0+46) (build 10+46)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (10+46, mixed mode, tiered, compressed oops, g1 gc, windows-amd64)
# Problematic frame:
# C [avformat-57.dll+0xd8c19]
#
# No core dump will be written. Minidumps are not enabled by default on client versions of Windows
#
# An error report file with more information is saved as:
# E:\Users\knd20\IdeaProjects\comdekinci\hs_err_pid13448.log
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
ame : SoundHandler
[h264 @ 000001b7fda058a0] nal_unit_type: 7, nal_ref_idc: 3
[h264 @ 000001b7fda058a0] nal_unit_type: 8, nal_ref_idc: 3
[SWR @ 000001b7ff3f6c00] Using fltp internally between filters
[SWR @ 000001b7ff40bbc0] Using fltp internally between filters
[SWR @ 000001b7ff40bbc0] Matrix coefficients:
[SWR @ 000001b7ff40bbc0] FL: FL:1.000000 FR:0.000000 FC:0.707107 LFE:0.000000 BL:0.707107 BR:0.000000
[SWR @ 000001b7ff40bbc0] FR: FL:0.000000 FR:1.000000 FC:0.707107 LFE:0.000000 BL:0.000000 BR:0.707107
[h264 @ 000001b7fda058a0] nal_unit_type: 5, nal_ref_idc: 3
[h264 @ 000001b7fda058a0] Reinit context to 1280x720, pix_fmt: yuv420p
[libx264 @ 000001b7d700e2c0] frame= 0 QP=25.00 NAL=3 Slice:I Poc:0 I:3600 P:0 SKIP:0 size=120620 bytes
So, it's crashing in av_interleaved_write_frame(). It's probably getting called with invalid arguments, but the cause of the problem appears to be this:
[mp4 @ 000001b7d700cbe0] muxer does not support non seekable output
The RTMP protocol most likely cannot support the MP4 format. You should always avoid the MP4 format for streaming.
saudet
on 14 Apr 2018
Hi @DeKinci, Please send me correct Gradle in Android Studio that helps add overlay code working. Thank you.
Finally after some format filter expirements this works just fine:
public class TestFFmpeg {
private static final String LOG_TAG = TestFFmpeg.class.getSimpleName();
public void run() {
try {
Log.d(LOG_TAG, "Setting hardware");
File video = FileHelper.getFileFromAppDir("video.mp4");
String fontPath = FileHelper.getFileFromAppDir("font.ttf").getAbsolutePath();
String overlayPath = FileHelper.getFileFromAppDir("img.png").getAbsolutePath();
String output = FileHelper.createFileInAppDir("test" + System.currentTimeMillis() / 1000 + ".mp4").getAbsolutePath();
String filterString = "movie=" + overlayPath + "[img];" +
"[in][img]overlay,format=pix_fmts=bgr24," +
"drawtext=fontfile=" + fontPath + ":text='TEST TEST TEST TEST'";
FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(video);
FFmpegFrameFilter filter = new FFmpegFrameFilter(filterString, 1280, 720);
FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(output, 1280, 720, 2);
setUpRecorder(recorder);
Log.d(LOG_TAG, "Starting recording");
recorder.start();
filter.start();
grabber.start();
long startTime = System.currentTimeMillis();
Frame grabbedFrame;
Frame filteredFrame;
Log.d(LOG_TAG, "Recording started");
while ((grabbedFrame = grabber.grab()) != null) {
Log.v(LOG_TAG, "New grabbed frame");
filter.push(grabbedFrame);
while ((filteredFrame = filter.pull()) != null) {
Log.v(LOG_TAG, "New filtered frame");
long t = 1000 * (System.currentTimeMillis() - startTime);
if (t > recorder.getTimestamp())
recorder.setTimestamp(t);
recorder.record(filteredFrame);
}
}
int elapsedSeconds = (int) (System.currentTimeMillis() - startTime) / 1000;
Log.d(LOG_TAG, "Finishing recording in " + elapsedSeconds / 60 + "m " + elapsedSeconds % 60 + "s");
grabber.stop();
filter.stop();
recorder.stop();
Log.d(LOG_TAG, "Recording finished");
} catch (Exception e) {
e.printStackTrace();
}
}
private void setUpRecorder(FFmpegFrameRecorder recorder) {
recorder.setInterleaved(true);
recorder.setFormat("mp4");
recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);
recorder.setFrameRate(30);
recorder.setGopSize(30 * 2);
recorder.setVideoBitrate(3_000_000);
recorder.setVideoOption("tune", "zerolatency");
recorder.setVideoOption("preset", "ultrafast");
recorder.setVideoOption("crf", "28");
recorder.setAudioCodec(avcodec.AV_CODEC_ID_AAC);
recorder.setSampleRate(44_100);
recorder.setAudioQuality(0);
recorder.setAudioChannels(2);
recorder.setAudioBitrate(128_000);
recorder.setAudioOption("crf", "0");
}
}
myScoopAndroidCamera
on 17 Apr 2018
@myScoopAndroidCamera I am currently using manual installation of JavaCV (copy needed jars to libs folder, extract .so files from used jars to src/main/jniLibs or any other folder, but you will need to specify it in gradle). I am using the latest version of javacv, which is 1.4.2-SNAPSHOT. This is build.gradle:
apply plugin: 'com.android.application'
android {
compileSdkVersion 27
defaultConfig {
applicationId "com.domain.appname"
minSdkVersion 23
targetSdkVersion 27
versionCode 29
versionName "0.3.1"
multiDexEnabled true
testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
ndk {
//Now more platforms can be used
abiFilters "armeabi-v7a", "x86"
}
}
buildTypes {
release {
minifyEnabled true
proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
}
}
configurations.all {
resolutionStrategy.force 'com.google.code.findbugs:jsr305:1.3.9'
}
compileOptions {
sourceCompatibility JavaVersion.VERSION_1_8
targetCompatibility JavaVersion.VERSION_1_8
}
buildToolsVersion '27.0.3'
}
repositories {
jcenter()
maven { url 'https://jitpack.io' }
maven { url 'https://oss.sonatype.org/content/repositories/snapshots' }
}
dependencies {
implementation fileTree(include: ['*.jar'], dir: 'libs')
//noinspection GradleCompatible
implementation 'com.android.support:appcompat-v7:27.1.1'
implementation 'com.android.support.constraint:constraint-layout:1.0.2'
implementation 'com.android.support:support-v4:27.1.1'
implementation 'com.android.support:design:27.1.1'
testImplementation 'junit:junit:4.12'
androidTestImplementation 'com.android.support.test:runner:1.0.1'
androidTestImplementation 'com.android.support.test.espresso:espresso-core:3.0.1'
//Other libs
implementation 'com.getkeepsafe.relinker:relinker:1.2.3'
implementation files('libs/javacv.jar')
}
Also I use relinker to get rid of unsatisfied link errors.
DeKinci
on 17 Apr 2018
👍1
Works fine thanks!
DeKinci
on 17 Apr 2018
Great! What did you do to make it work in the end?
saudet
on 18 Apr 2018
It was the mp4 format.
I tested it in flv in different situations and everything works fine.
Another problem I bumped in: dynamic text (added via drawtext textfile) will not be changed, when applied to image, which is an overlay for the video. But search says it is ffmpeg related issue (can be solved with two inports configuration, which AFAIK is not supported in javacv) so it is hard to say whether it is an issue or not.
DeKinci
on 18 Apr 2018
Ah yes, we would need to add support for multiple inputs. This is covered by issue #955 and can be implemented. We just need someone to help out with the task. :)
saudet
on 18 Apr 2018
Hi @saudet, When you will release official JavaCV version 1.4.2?
myScoopAndroidCamera
on 18 Apr 2018
@myScoopAndroidCamera Not decided yet, but now that we can use the CI servers to do releases, we can start setting up a schedule. Once every quarter (3 months) sounds like a good cycle, and that would mean end of June...
In the meantime, snapshots are available, so please use them: http://bytedeco.org/builds/
saudet
on 18 Apr 2018
👍1
It works fine. Thank you so much :)
implementation group: 'org.bytedeco', name: 'javacv', version: '1.4.2-SNAPSHOT'
implementation group: 'org.bytedeco.javacpp-presets', name: 'opencv', version: '3.4.1-1.4.1', classifier: 'android-arm'
implementation group: 'org.bytedeco.javacpp-presets', name: 'ffmpeg', version: '3.4.2-1.4.1', classifier: 'android-arm'
String filterString = "movie=" + overlayPath + "[img];" +
"[in][img]overlay,format=pix_fmts=bgr24";
Filter works fine but can we change filters dynamically?
For example can we change text while recording?
I tried to set new string of filters before every frame but it doesn't effect anything:
frameFilter.setFilters(filterString);
frameFilter.push(recordedFrame.getFrame());
anonym24
on 15 Jul 2018
@anonym24, you can draw text from file and update it (atomically!). Also set the reload flag. If you need something more versatile, like changing color or font size, you can stop pushing frames to filter, wait until all filter frames are pulled, and then swap filter for the one you need
DeKinci
on 15 Jul 2018
@DeKinci yes it's ok for predefined subtitles
actually I want to record current date (yyyy.mm.dd hh:mm:ss) but I guess this task isn't for FFmpeg
I already tried MediaRecorderwith Surface source but it's very buggy (doesn't work for all devices): https://stackoverflow.com/questions/51332386/mediarecorder-and-videosource-surface-stop-failed-1007-a-serious-android-bug
I guess the only is to use MediaCodecand scary OpenGL (GLES) stuff to process frames while recording (put text or images), it's very difficult to understand and implement
anonym24
on 15 Jul 2018
@anonym24, I used ffmpeg to create a scoreboard overlay while streaming, so it is possible to implement time and date. You actually can change text in the file on the fly and with that change the written text. The only necessary thing is to update file atomically (which is trivial: do it synchronously between editing frames). You can read more about it in ffmpeg docs for drawtext filter
DeKinci
on 15 Jul 2018
@DeKinci ffmpeg for Android is bad in performance anyway, it lugs when recording 1080p video
I also used to use OpenCV to edit frames, but it also takes some time to convert byte[] to Mat, process it and then again to byte[]
anonym24
on 15 Jul 2018
FFmpeg now supports MediaCodec acceleration thanks to @tmm1, see commit https://github.com/bytedeco/javacpp-presets/commit/075c29ea2b2896058bc0437c8fea1d2edf6a1165, so that should be pretty fast now.
And to edit with OpenCV, we don't need to copy the frames, use OpenCVFrameConverter to access the raw data easily and efficiently.
saudet
on 16 Jul 2018
I haven't looked at JavaCV code in detail, but I think FFmpegFrameGrabber would still need to be taught to use MediaCodec (via avcodec_find_decoder_by_name("h264_mediacodec")).
This would use the hardware accelerated decoder, but still copy back the pixel data to memory so you can run all the regular ffmpeg filters.
There are also more complex ways of using the decoder to return opaque Surface-backed buffers, or to render to a SurfaceTexture (to convert the video frame into an OpenGL texture). For these, a Surface has to be created in Java first and passed to libavcodec via AVMediaCodecDeviceContext.
Oh, and you also have to call av_jni_set_java_vm to pass down the JavaVM* into ffmpeg during initialization.
tmm1
on 16 Jul 2018
@tmm1 Right, let's just get this done. I've added support for that in commit https://github.com/bytedeco/javacv/commit/a17a934cc71dba903784ff7f9370d2ae5634673e.
FFmpeg doesn't export an JNI_OnLoad()? IMO, that would make more sense than having to call an init function...
saudet
on 16 Jul 2018
I agree that would make more sense. I can propose a patch for ffmpeg 4.1 to do it automatically.
(There may be some users who compile libavcodec statically and combine it with other archives into a single shared library, so perhaps exporting the global JNI_OnLoad could conflicts.)
Surprisingly this wasn't really discussed on the ffmpeg list when the api was implemented: https://ffmpeg.org/pipermail/ffmpeg-devel/2016-March/190546.html
cc @mbouron
tmm1
on 16 Jul 2018
👍1
Yeah, it wouldn't work too well for static builds...
saudet
on 16 Jul 2018
@saudet
I use JavaCV FFmpeg with original OpenCV4Andoid
We still have to convert it to RGBA to be able to draw color text
If we use OpenCV putText on YuvFrame we can only draw black/white/gray
// init once
mRgba = new Mat();
mYuvFrameData= new Mat(mFrameHeight + (mFrameHeight/2), mFrameWidth, CvType.CV_8UC1);
// do next in onPreviewFrame(byte[] data, ...)
mYuvFrameData.put(0, 0, data);
Imgproc.cvtColor(mYuvFrameData, mRgba, Imgproc.COLOR_YUV2RGBA_NV21, 4);
// draw something on mRgba using putText method
if (mByte == null) mByte = new byte[(int)(mRgba.total()) * mRgba.channels()];
mRgba.get(0, 0, mByte);
Does JavaCV already supports FFmpegFrameRecorder with MediaCodec?
anonym24
on 16 Jul 2018
Yes, that's true OpenCV doesn't support YUV. But drawText does? Good to know.
According to @tmm1 yes it does.
saudet
on 16 Jul 2018
FFmpeg Filter's drawText?
I think it can (I tried only white color, "drawtext=fontfile=" + fontPath + ":text='TEST': fontcolor=white: fontsize=96"), but I need dynamic text and it's much easier to change text with OpenCV before encoding each frame, so I decided not to use FFmpeg fitlers
@tmm1 will there be a little documentation how to use MediaCodec with FFmpegFrameRecorder?
anonym24
on 16 Jul 2018
seems OpenCV doesn't directly put text to data (putmethod makes copy)
mYuvFrameData.put(0, 0, data);
Imgproc.putText(mYuvFrameData, "test", mDatePosition, Core.FONT_HERSHEY_SIMPLEX,
0.8, mColorBlack, 5, Core.LINE_AA, false);
so when using such we still have to get new byte and use this byte and not old byte from onPreviewCallback
if (mByte == null) mByte = new byte[(int)(mYuvFrameData.total()) * mYuvFrameData.channels()];
mYuvFrameData.get(0, 0, mByte);
How does JavaCV OpenCVFrameConverter work? Does it create Mat from Frame and doesn't copy anything?
If so mb I will use it even if it allows only black/white text
Also can I do it with OpenCV4Android?
anonym24
on 16 Jul 2018
Yes, that's how it works. We can access the data pointer from the Java bindings of OpenCV as well: https://github.com/bytedeco/javacpp/issues/38#issuecomment-140728812
saudet
on 16 Jul 2018
@saudet but actually copying/converting of frames (bytes, mats) isn't that big problem in video recording
because in next example (StackOverflow) with MediaRecorder I converted byte[] to Mat and then Mat to Btmap and then used that Bitmap to draw on MediaRecorder's Surface (lockCanvas)
and fps of video was pretty good (smooth enough, no lugs, even with 1080p)
but it doesn't work on all devices
so it depends on how fast recorder can encode frames, I hope FFmpeg + MediaRecorder will work much better
anonym24
on 16 Jul 2018
I didn't test it, but something like this should work with JavaCV 1.4.2 now:
FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(...);
grabber.setVideoCodecName("h264_mediacodec");
...
Tried this
mFrameRecorder = new FFmpegFrameRecorder(mVideoFilePath, videoWidth, videoHeight, 1);
mFrameRecorder.setFormat("mp4");
mFrameRecorder.setVideoCodecName("h264_mediacodec");
weird but it used "MPEG-4 part 2" codec
anonym24
on 18 Jul 2018
@anonym24 What does FFmpeg output to the log?
saudet
on 18 Jul 2018
avutil.av_log_set_level(avutil.AV_LOG_VERBOSE);
FFmpegLogCallback.set();
Logs during start and stop FFmpegFrameRecorder
07-18 13:30:55.343 20877-20877/*** I/System.out: Output #0, mp4, to '/storage/A54E-14E9/Android/data/***/files/test.mp4':
07-18 13:30:55.344 20877-20877/*** I/System.out: Output #0, mp4, to '/storage/A54E-14E9/Android/data/***/files/test.mp4':
Metadata:
Metadata:
encoder :
07-18 13:30:55.345 20877-20877/*** I/System.out: encoder :
Lavf58.12.100
Lavf58.12.100
07-18 13:30:55.346 20877-20877/*** I/System.out: Stream #0:0
Stream #0:0
07-18 13:30:55.347 20877-20877/*** I/System.out: : Video: mpeg4, 1 reference frame (mp4v / 0x7634706D), yuv420p, 1280x720 (0x0), q=2-31, 400 kb/s
: Video: mpeg4, 1 reference frame (mp4v / 0x7634706D), yuv420p, 1280x720 (0x0), q=2-31, 400 kb/s
,
,
12288 tbn
07-18 13:30:55.348 20877-20877/*** I/System.out: 12288 tbn
07-18 13:30:55.659 20877-21159/*** V/RenderScript: 0x55cfffc700 Launching thread(s), CPUs 8
when I use mFrameRecorder.setVideoCodec(avcodec.AV_CODEC_ID_H264); there is much more logs than when using mFrameRecorder.setVideoCodecName("h264_mediacodec");
anonym24
on 18 Jul 2018
What does avcodec_find_decoder_by_name("h264_mediacodec")) return?
saudet
on 18 Jul 2018
avcodec.AVCodec avCodec = avcodec.avcodec_find_decoder_by_name("h264_mediacodec");
if (avCodec != null) {
Log.i(TAG, "avCodec != null, " + avCodec.name().getString() + " " + avCodec.long_name().getString());
} else {
Log.i(TAG, "avCodec == null");
}
output
avCodec != null, h264_mediacodec H.264 Android MediaCodec decoder
anonym24
on 18 Jul 2018
Ok, looks good, what about avcodec_find_encoder_by_name("h264_mediacodec"))?
saudet
on 18 Jul 2018
here's encoder:
avcodec.AVCodec avCodec = avcodec.avcodec_find_encoder_by_name("h264_mediacodec");
if (avCodec != null) {
Log.i(TAG, "avCodec != null, " + avCodec.name().getString() + " " + avCodec.long_name().getString());
} else {
Log.i(TAG, "avCodec == null");
}
it returns null
avCodec == null
anonym24
on 18 Jul 2018
Hum, looks like encoding isn't supported:
https://trac.ffmpeg.org/wiki/HWAccelIntro
saudet
on 18 Jul 2018
Yes it looks like mediacodec encoders do not exist in ffmpeg. It wouldn't be very hard to add, but no one has submitted any patches so far.
tmm1
on 18 Jul 2018
@tmm1 can you do it? (if you're interested of course)
I don't really work with C or C++, it's not my subject
anonym24
on 18 Jul 2018
BTW, I've implemented support for multiple inputs in commit https://github.com/bytedeco/javacv/commit/3c92eef6e3aa21eca170fc8022fd41254cfcc452, so we can now use a filter graph like "[0][1]overlay", although I've decided to follow the [0:v] ... [v], etc as convention for the names of the inputs and outputs as it's really not clear how the ffmpeg program comes up with the names it uses: https://github.com/bytedeco/javacv/blob/master/platform/src/test/java/org/bytedeco/javacv/FrameFilterTest.java#L163
Please give it a try with the snapshots: http://bytedeco.org/builds/ Thanks!
saudet
on 28 Oct 2018
It's great, now it is possible to create proper slow-mo filter and others with splitted audio and video editing without manually splitting streams. Thanks!
DeKinci
on 30 Oct 2018
I want to overlay two seperate PNG files on the live video stream at the same time. I have managed to overlay a single image on the video using the movie filter.
How do I add the second image at the same time?
Thanks
davepaiva
on 20 Dec 2019
You can chain filters together, just direct first filter output to second's input.
[in]filter1[filtered];[filtered]filter2[out]
Also FYI there is the overlay filter and it works well for overlaying pngs on the live stream
https://ffmpeg.org/ffmpeg-filters.html#overlay-1
DeKinci
on 20 Dec 2019
👍2
Thanks @DeKinci chaining filters together worked for me !
davepaiva
on 20 Dec 2019
Was this page helpful?
0 / 5 - 0 ratings
Related issues
iamazy
路
4Comments
Maleandr
路
3Comments
Bahramudin
路
3Comments
chrisliu12345
路
4Comments
Bahramudin
路
3Comments
Most helpful comment
You can chain filters together, just direct first filter output to second's input.
[in]filter1[filtered];[filtered]filter2[out]Also FYI there is the overlay filter and it works well for overlaying pngs on the live stream
https://ffmpeg.org/ffmpeg-filters.html#overlay-1