Srt: stransmit does not work with UDP input (it works with pipes)

Created on 1 Sep 2017 · 19Comments · Source: Haivision/srt

Hi, thanks a lot to open source this work, it is very interestig option for people who work in the live streaming maket.
But I compiled this code in a Docker (Ubuntu 16 first, and Ubuntu 14 after) and I could NOT make it work (corrupted output).
Thinking that the problem could have been the docker I compiled again in a VM Cent OS 6 (VMWare), and the following test resulted in the same corrupted output that I saw in my docker experiment.

CentOS test:

SOURCE (terminal1):

ffmpeg -f lavfi -re -i smptebars=duration=60:size=1280x720:rate=30 -f lavfi -re -i sine=frequency=1000:duration=60:sample_rate=44100 -pix_fmt yuv420p -c:v libx264 -b:v 1000k -g 30 -keyint_min 120 -profile:v baseline -preset veryfast -c:a libfdk_aac -b:a 96k -f mpegts udp://127.0.0.1:5000

SRT TX (terminal2):

./stransmit -r:10 -s:5 udp://:5000 srt://:9000

SRT RX & play (terminal3):

./stransmit srt://127.0.0.1:9000 file://con | ffplay -

This is a sample of what ffplay claims:

[aac @ 0x7f3d5c006f20] decode_pce: Input buffer exhausted before END element found
[aac @ 0x7f3d5c006f20] channel element 1.6 is not allocated f=0/0   
[aac @ 0x7f3d5c006f20] channel element 2.5 is not allocated
[aac @ 0x7f3d5c006f20] Sample rate index in program config element does not match the sample rate index configured by the container.
[aac @ 0x7f3d5c006f20] decode_pce: Input buffer exhausted before END element found
[mpegts @ 0x7f3d5c0008c0] PES packet size mismatchsq=    0B f=0/0   
[aac @ 0x7f3d5c006f20] Prediction is not allowed in AAC-LC.
[aac @ 0x7f3d5c006f20] channel element 1.6 is not allocated
[aac @ 0x7f3d5c006f20] channel element 2.5 is not allocated
[mpegts @ 0x7f3d5c0008c0] PES packet size mismatchsq=    0B f=0/0   
[aac @ 0x7f3d5c006f20] channel element 3.12 is not allocated
[aac @ 0x7f3d5c006f20] SBR was found before the first channel element.
[aac @ 0x7f3d5c006f20] channel element 2.10 is not allocated
[aac @ 0x7f3d5c006f20] channel element 1.6 is not allocated
[mpegts @ 0x7f3d5c0008c0] PES packet size mismatchsq=    0B f=0/0   
[aac @ 0x7f3d5c006f20] channel element 2.13 is not allocated
[h264 @ 0x7f3d5c16c5e0] left block unavailable for requested intra mode at 0 27
[h264 @ 0x7f3d5c16c5e0] error while decoding MB 0 27
[h264 @ 0x7f3d5c16c5e0] concealing 1489 DC, 1489 AC, 1489 MV errors in I frame
[aac @ 0x7f3d5c006f20] channel element 2.4 is not allocated
[aac @ 0x7f3d5c006f20] channel element 3.4 is not allocated
[mpegts @ 0x7f3d5c0008c0] PES packet size mismatchsq=    0B f=0/0   
[aac @ 0x7f3d5c006f20] skip_data_stream_element: Input buffer exhausted before END element found
[aac @ 0x7f3d5c006f20] If you heard an audible artifact, there may be a bug in the decoder. Clipped noise gain (-172 -> -100) is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[aac @ 0x7f3d5c006f20] If you want to help, upload a sample of this file to ftp://upload.ffmpeg.org/MPlayer/incoming/ and contact the ffmpeg-devel mailing list.
[aac @ 0x7f3d5c006f20] If you heard an audible artifact, there may be a bug in the decoder. Clipped noise gain (-175 -> -100) is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[aac @ 0x7f3d5c006f20] If you want to help, upload a sample of this file to ftp://upload.ffmpeg.org/MPlayer/incoming/ and contact the ffmpeg-devel mailing list.
[aac @ 0x7f3d5c006f20] If you heard an audible artifact, there may be a bug in the decoder. Clipped noise gain (-175 -> -100) is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[aac @ 0x7f3d5c006f20] If you want to help, upload a sample of this file to ftp://upload.ffmpeg.org/MPlayer/incoming/ and contact the ffmpeg-devel mailing list.
[aac @ 0x7f3d5c006f20] If you heard an audible artifact, there may be a bug in the decoder. Clipped noise gain (-177 -> -100) is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[aac @ 0x7f3d5c006f20] If you want to help, upload a sample of this file to ftp://upload.ffmpeg.org/MPlayer/incoming/ and contact the ffmpeg-devel mailing list.
[aac @ 0x7f3d5c006f20] SSR is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[aac @ 0x7f3d5c006f20] If you want to help, upload a sample of this file to ftp://upload.ffmpeg.org/MPlayer/incoming/ and contact the ffmpeg-devel mailing list.
[aac @ 0x7f3d5c006f20] channel element 2.5 is not allocated
[aac @ 0x7f3d5c006f20] skip_data_stream_element: Input buffer exhausted before END element found
[mpegts @ 0x7f3d5c0008c0] PES packet size mismatchsq=    0B f=0/0   
[aac @ 0x7f3d5c006f20] Sample rate index in program config element does not match the sample rate index configured by the container.
[aac @ 0x7f3d5c006f20] decode_pce: Input buffer exhausted before END element found
[aac @ 0x7f3d5c006f20] channel element 3.4 is not allocated
[mpegts @ 0x7f3d5c0008c0] PES packet size mismatchsq=    0B f=0/0   
[aac @ 0x7f3d5c006f20] Reserved bit set.
[aac @ 0x7f3d5c006f20] channel element 3.4 is not allocated
    Last message repeated 2 times
[mpegts @ 0x7f3d5c0008c0] PES packet size mismatchsq=    0B f=0/0   
[aac @ 0x7f3d5c006f20] Prediction is not allowed in AAC-LC.
[aac @ 0x7f3d5c006f20] channel element 1.6 is not allocated
[aac @ 0x7f3d5c006f20] skip_data_stream_element: Input buffer exhausted before END element found
[mpegts @ 0x7f3d5c0008c0] PES packet size mismatchsq=    0B f=0/0   
[aac @ 0x7f3d5c006f20] skip_data_stream_element: Input buffer exhausted before END element found
[aac @ 0x7f3d5c006f20] channel element 2.4 is not allocated
[h264 @ 0x7f3d5c16c5e0] left block unavailable for requested intra mode at 0 27
[h264 @ 0x7f3d5c16c5e0] error while decoding MB 0 27
[h264 @ 0x7f3d5c16c5e0] concealing 1489 DC, 1489 AC, 1489 MV errors in I frame
[aac @ 0x7f3d5c006f20] channel element 2.13 is not allocatedf=0/0   
    Last message repeated 1 times
[mpegts @ 0x7f3d5c0008c0] PES packet size mismatchsq=    0B f=0/0   
[aac @ 0x7f3d5c006f20] skip_data_stream_element: Input buffer exhausted before END element found
[aac @ 0x7f3d5c006f20] Inconsistent channel configuration.
[aac @ 0x7f3d5c006f20] get_buffer() failed
[aac @ 0x7f3d5c006f20] channel element 3.4 is not allocated
[aac @ 0x7f3d5c006f20] Prediction is not allowed in AAC-LC.
[mpegts @ 0x7f3d5c0008c0] PES packet size mismatchsq=    0B f=0/0   
[aac @ 0x7f3d5c006f20] channel element 3.6 is not allocated
[aac @ 0x7f3d5c006f20] channel element 2.13 is not allocatedf=0/0   
    Last message repeated 1 times
[aac @ 0x7f3d5c006f20] Sample rate index in program config element does not match the sample rate index configured by the container.
[aac @ 0x7f3d5c006f20] decode_pce: Input buffer exhausted before END element found
[mpegts @ 0x7f3d5c0008c0] PES packet size mismatch
[aac @ 0x7f3d5c006f20] decode_pce: Input buffer exhausted before END element found
[aac @ 0x7f3d5c006f20] channel element 2.4 is not allocated f=0/0   
[aac @ 0x7f3d5c006f20] channel element 3.4 is not allocated
[h264 @ 0x7f3d5c1652c0] left block unavailable for requested intra mode at 0 27
[h264 @ 0x7f3d5c1652c0] error while decoding MB 0 27
[h264 @ 0x7f3d5c1652c0] concealing 1489 DC, 1489 AC, 1489 MV errors in I frame
[mpegts @ 0x7f3d5c0008c0] PES packet size mismatchsq=    0B f=0/0   
[aac @ 0x7f3d5c006f20] Sample rate index in program config element does not match the sample rate index configured by the container.
[aac @ 0x7f3d5c006f20] decode_pce: Input buffer exhausted before END element found
[mpegts @ 0x7f3d5c0008c0] PES packet size mismatchsq=    0B f=0/0   
[aac @ 0x7f3d5c006f20] Sample rate index in program config element does not match the sample rate index configured by the container.
[aac @ 0x7f3d5c006f20] decode_pce: Input buffer exhausted before END element found
[aac @ 0x7f3d5c006f20] TYPE_FIL: Input buffer exhausted before END element found
[aac @ 0x7f3d5c006f20] channel element 2.13 is not allocated
[aac @ 0x7f3d5c006f20] Sample rate index in program config element does not match the sample rate index configured by the container.
[aac @ 0x7f3d5c006f20] decode_pce: Input buffer exhausted before END element found

PS : I also tried to save the file and play it later, and I also tried to send it to UDP receiving it in ffmpeg. All experiments same result: Corrupted output

Question

Source

jordicenzano

All 19 comments

What happens if you use two different machines in the same local network, instead of streaming through SRT within the range of the same machine? If still the same, what happens if you send packets to another machine for ffplay through UDP directly?

Just as a reminder: stransmit uses by default the "live" transfer mode and it's kinda matter of trust as to whether the input data really IS a live video stream.

ethouris on 1 Sep 2017

1- Sending UDP inside the same machine using just ffmpeg works perfectly:

Source (terminal1) - same as SRT source:

ffmpeg -f lavfi -re -i smptebars=duration=60:size=1280x720:rate=30 -f lavfi -re -i sine=frequency=1000:duration=60:sample_rate=44100 -pix_fmt yuv420p -c:v libx264 -b:v 1000k -g 30 -keyint_min 120 -profile:v baseline -preset veryfast -c:a libfdk_aac -b:a 96k -f mpegts udp://127.0.0.1:5000

Destination (terminal2) - perfect playback:

ffplay udp://0.0.0.0:5000

2- The input data is a live video stream, look ffmpeg params -f lavfi -re -i smptebars=duration=60:size=1280x720:rate=30 -f lavfi -re -i sine=frequency=1000:duration=60:sample_rate=44100 -pix_fmt yuv420p -c:v libx264 -b:v 1000k -g 30 -keyint_min 120 -profile:v baseline -preset veryfast -c:a libfdk_aac -b:a 96k -f mpegts udp://127.0.0.1:5000

3- Not tested SRT across machines, theoretically should not matter

jordicenzano on 1 Sep 2017

This matters even theoretically. At least SRT isn't predicted to work correctly within the frames of a single system.

You can try to see what happens if you pass the data from ffmpeg to stransmit directly though stdout-stdin.

ethouris on 1 Sep 2017

Tested between 2 different computers both MACOS + VMWare running CentOS6

Computer1 (terminal1) 192.168.1.48:

ffmpeg -f lavfi -re -i smptebars=duration=60:size=1280x720:rate=30 -f lavfi -re -i sine=frequency=1000:duration=60:sample_rate=44100 -pix_fmt yuv420p -c:v libx264 -b:v 1000k -g 30 -keyint_min 120 -profile:v baseline -preset veryfast -c:a libfdk_aac -b:a 96k -f mpegts udp://127.0.0.1:5000

Computer1 (terminal2) 192.168.1.48:

./stransmit -v udp://:5000 "srt://192.168.1.48:9000?mode=server"

Computer2 (terminal1) 192.168.1.129:

./stransmit "srt://192.168.1.48:9000?mode=client" "file://con" | ffplay -

RESULT: CORRUPTED PLAYBACK

jordicenzano on 1 Sep 2017

The last test you asked me (using only pipes) worked PERFECTLY!!

Computer 1:

ffmpeg -f lavfi -re -i smptebars=duration=60:size=1280x720:rate=30 -f lavfi -re -i sine=frequency=1000:duration=60:sample_rate=44100 -pix_fmt yuv420p -c:v libx264 -b:v 1000k -g 30 -keyint_min 120 -profile:v baseline -preset veryfast -c:a libfdk_aac -b:a 96k -f mpegts - |  ./stransmit -v "file://con" "srt://192.168.1.48:9000?mode=server"

Computer 2:

./stransmit "srt://192.168.1.48:9000?mode=client" "file://con" | ffplay -

Also the following test using ONLY pipes in the SAME computer worked perfectly (as I expected):
Computer 1:

ffmpeg -f lavfi -re -i smptebars=duration=60:size=1280x720:rate=30 -f lavfi -re -i sine=frequency=1000:duration=60:sample_rate=44100 -pix_fmt yuv420p -c:v libx264 -b:v 1000k -g 30 -keyint_min 120 -profile:v baseline -preset veryfast -c:a libfdk_aac -b:a 96k -f mpegts - |  ./stransmit -v "file://con" "srt://:9000?mode=server"

Computer 1:

./stransmit "srt://127.0.0.1:9000?mode=client" "file://con" | ffplay -

Conclusion: You have a problem with UDP input (probably output too) that makes the utility stransmit unusable in a real network scenario (for me just works with pipes). Remember the following test using only ffmpeg and UDP woks always perfectly:

#TX:
ffmpeg -f lavfi -re -i smptebars=duration=60:size=1280x720:rate=30 -f lavfi -re -i sine=frequency=1000:duration=60:sample_rate=44100 -pix_fmt yuv420p -c:v libx264 -b:v 1000k -g 30 -keyint_min 120 -profile:v baseline -preset veryfast -c:a libfdk_aac -b:a 96k -f mpegts udp://127.0.0.1:5000

#RX
ffplay udp://0.0.0.0:5000

Anyway, @ethouris thanks a lot for your fast and precise help. I think this could be a very good tool but probably needs a little bit of more work

jordicenzano on 1 Sep 2017

As I said, SRT isn't predicted to transmit a network stream within one operating system.

In order to know what happened to the UDP packets and at which stage they are being lost, you'd need pcaps recorded from all ports separately for the whole session. I wouldn't bet from upside that it's SRT that loses them.

ethouris on 1 Sep 2017

Maybe you can have a look at the loopback interface stats after using your non-working scenario and see if UDP packets were dropped. I am not familiar with ffmpeg: which encoding bitrate are you using?

jeandube on 1 Sep 2017

Make sure you don't have exiting UDP traffic on port 9000. You can either sniff that port or retry your ffmpeg test on port 9000.

midicase on 1 Sep 2017

Thanks @midicase , since the test with SRT with pipes (in and out) worked properly in port 9000 I do not think that was the case.
Anyway:
1- Repeated ffmpeg test in port 9000: WORKS GOOD as expected.

#TX
ffmpeg -f lavfi -re -i smptebars=duration=60:size=1280x720:rate=30 -f lavfi -re -i sine=frequency=1000:duration=60:sample_rate=44100 -pix_fmt yuv420p -c:v libx264 -b:v 1000k -g 30 -keyint_min 120 -profile:v baseline -preset veryfast -c:a libfdk_aac -b:a 96k -f mpegts udp://127.0.0.1:9000

#RX:
ffplay udp://0.0.0.0:9000

jordicenzano on 2 Sep 2017

Hi @jeandube , those are the netstat -s results before and after a failed test. I can not see anything relevant.

# BEFORE the test:

Udp:
    33551 packets received
    330 packets to unknown port received.
    0 packet receive errors
    30424 packets sent

#AFTER THE 60s test (with corrupted playback):
Udp:
    42666 packets received
    330 packets to unknown port received.
    0 packet receive errors
    39539 packets sent

PS: I'm using 1Mbps for video (I tested 3Mbps too with the same results)

jordicenzano on 2 Sep 2017

Not sure if this matters in this case, but remember that stransmit reads - from whatever source - in 1316-byte pieces.

ethouris on 5 Sep 2017

@ethouris Thanks a lot! This addition fixes everything pkt_size=1316, so the final command that make it work with UDP and ffmpeg is:

ffmpeg -f lavfi -re -i smptebars=duration=60:size=1280x720:rate=30 -f lavfi -re -i sine=frequency=1000:duration=60:sample_rate=44100 -pix_fmt yuv420p -c:v libx264 -b:v 1000k -g 30 -keyint_min 120 -profile:v baseline -preset veryfast -c:a libfdk_aac -b:a 96k -f mpegts "udp://127.0.0.1:5000?pkt_size=1316"

I still think that there is a bug here, in my opinion stransmit should be resilient to any UDP packet size. Or at least mention in the readme that it only allows 1316 bytes of payload in UDP packets.

Thanks again @ethouris

jordicenzano on 5 Sep 2017

👍1

Yes, SRT needs fixes in description and also it should take care of checking the input size in particular mode. I'm working on that currently. Actually the maximum possible payload size for a single UDP packet in SRT is 1456, and there's nothing that can be done about this (it's 1500 standard MTU size, decreased by 28, which is ETH+IP+UDP header, and 16 which is SRT header). Possibly when you did not put a limit on a size, it sent 1472 in a single UDP packet, and there's no way for SRT to transport it.

1316 is 188*7, and why 188 it's best explained here: https://en.wikipedia.org/wiki/MPEG_transport_stream

That's the default "chunk size" used by stransmit. You can override this value by using -c option. But still, only up to 1456.

In "live" mode (which is currently the only available in SRT) we utilize the "message mode" from UDT (UDT names it "datagram", which is extremely confusing), but the size of a single message shall not exceed the size that fits in one UDP packet (1456 bytes). This is because the best way to maintain the low latency is to keep data pieces as small as possible and rely on the ingest timing with data delivery (there are quite big differences between application's times when processing small and large portion of data at once, and every increased time spent during processing may worsen the delivery time and challenge the latency).

ethouris on 6 Sep 2017

@ethouris Thanks or your detailed explanation. According to your explanation looks like you use the UDP packet payload from input and you just "copy" the data to SRT.
Don't you think could be a good idea implement simple buffer in UDP input to isolate the UDP input packet sizes from SRT. Something like:

SOURCE -- UDP --> BUFFER (size = SRT configured packet size) --> SRT Coder --> .....

This approach should be the same (or very similar) as pipes, and it will make stransmit resilient to any input packet size. Also the latency introduced by a buffer of 1000 bytes (approx.) will be very low in any video transmission (if the stream is 500Kbps, very bad case, the added latency will be 16ms)

I'm not familiar at all with SRT architecture. So perhaps what I'm saying is not possible.

jordicenzano on 6 Sep 2017

The idea of live transport in SRT relies on the fact that UDP provides not only data, but also timing.

If I pass a single UDP packet for SRT transport, which contains exactly the data copied from the UDP packet, then the time to send the packet in SRT corresponds with the time when the UDP packet with this data was received. This is a key feature and actually the reason of creating this library. Without having the accurate times when receiving a single UDP packet, this whole solution loses any reason.

The key thing here is that when a video frame is being sent - for example - then it's split into single packets. This frame has its duration, as well as there are also other types of frames that should also occur in this time range. In result we have a time range that starts with the frame's DTS and ends with its duration, and this time range embraces this video frame and usually also some other data (like audio frames) that fit also in this time range. Now everything that is assigned as a data for that time range must be evenly split into single UDP packets. UDP packets, however, are not just stowed away in the beginning of this range with some "wait time" added at the end of this period. It's that the whole time period is split into single pieces in such a number as the UDP packets and every UDP packet has its own "duration" assigned, so the time not spent with sending this UDP packet is a single wait time per UDP packet.

This breakdown into single timed UDP packets must be done by the content-gnostic application because it must be done with the knowledge of timestamps in the frames. SRT is by definition content-agnostic and therefore it has completely no information that can be used for deciding what time gaps it should put between sending single UDP packets. In other words, it's not the role of SRT to split single "data packets" into UDP packets - "data packets" must fit in SRT payload size in live mode, and that's why SRT cannot work with "any" size of packets.

ethouris on 6 Sep 2017

Basically the payload size has to be smaller or equal to 1456 bytes. If you transport MPEG-TS content, we recommend to go with a multiple of the TS frame size. That's why we use 7 * 188 = 1316 bytes as our preferred payload size, which proves very efficient.

Of course, this has to be documented properly ...