Mpv: Synchronized playback over a network

[jdb-vsn]

Description source: http://www.mplayerhq.hu/DOCS/HTML/en/networksync.html

Multiple instances of MPlayer can synchronize playback over a network. This is useful for creating "video walls" with multiple screens controlled by different computers. Each MPlayer instance can play a different video, but they all will try to stay at the same time offset in the file. It is recommended but not necessary to encode the video files using the same codec and parameters.

The relevant options are -udp-master, -udp-slave, -udp-ip, -udp-port, and -udp-seek-threshold.

If -udp-master is given, MPlayer sends a datagram to -udp-ip (default: 127.0.0.1) on -udp-port (default: 23867) just before playing each frame. The datagram indicates the master's position in the file. If -udp-slave is given, MPlayer listens on -udp-ip/-udp-port and matches the master's position. Setting -udp-ip to the master's broadcast address allows multiple slaves having the same broadcast address to sync to the master. Note that this feature assumes an ethernet-like low-latency network connection. Your mileage may vary on high latency networks.

Mentioned on issue #898.

Also on https://github.com/mpv-player/mpv/issues/252#issuecomment-24864050.

[ghost]

Porting this back to mpv shouldn't be too hard. But I don't like the way mplayer integrates this (it looks like it blocks the player on network to achieve sync), so it should be done differently. The branch udp_sync contains an earlier attempt.

[nixternal]

I believe this is the only thing blocking the adoption of mvp over mplayer for a client project that relies heavily on mplayers udp sync stuff. I wish I had the time & knowledge to help here because I can honestly say mpv > * as a media player. I now use it over VLC.

[ghost]

The most annoying thing is actually that it'd require raw network programming, which is painful/unportable if it should work on windows.

[artwait]

I'm using the udp sync feature in mplayer right now for a video wall, but I'm running multiple slaves on a single machine (rather than across a network). The challenge with this is that the UDP port is blocked by the first slave, thus subsequent slaves are not able to listen. I've got a hacky work-around to this, but it would be nice to also have a solution that uses a non-network approach for sending timestamps between instances on a single machine. Just a thought.

[ghost]

it would be nice to also have a solution that uses a non-network approach for sending timestamps between instances on a single machine.

Do you have a specific mechanism in mind?

[lionep]

:+1:

[mgcrea]

:+1:

Would leveraging an external dependency like http://zeromq.org make it easier?
Otherwise, I would be fine with a linux-only support.

[gcoupelant]

:+1: This would really be awesome.

[smarek]

Hey guys, any progress on this feature please?

[ghost]

Nobody is working on it. It could now be implemented without mpv changes via one of the following mechanisms:

• as Lua script
• as a cplugin
• as a wrapper application using libmpv

[smarek]

@wm4 cool, so any network synchronized protocol and wrapper around mpv should suffice?
Maybe using --input-ipc-server and any daemon that converts network issued command into mpv json command (or just dumb pass-through) could work.

Also I've found this, but theres no documentation. https://github.com/derlaft/synco
Anybody tried to use that?

[rossy]

@smarek You could also look at http://syncplay.pl/

I don't know if it's accurate enough to use as a video wall (like MPlayer's UDP-sync apparently was,) but it definitely can sync multiple mpv instances over a network.

[Collocalini]

I've recently did just what Smarek offered - used --input-ipc-server mode to synchronize mpv and osmcplayer from kodi on raspberry pi. Supports play/pause detection. I used it mostly to show video from raspberry pi but take audio from a better audio card on a separate computer. Not perfect but I find it watchable most of the time.
mpvsync.zip

[bitbyt3r]

I've been trying to do this through libmpv and have had some success, however it isn't quite frame perfect. I'm using a loop similar to this:
`

 mpv_observe_property(player0, 0, "time-pos", MPV_FORMAT_DOUBLE);

 while (1) {
    mpv_event *event = mpv_wait_event(ctx[0], 1000);

    if (event->event_id == MPV_EVENT_PROPERTY_CHANGE) {
        mpv_event_property *prop = (mpv_event_property *)event->data;
        if (strcmp(prop->name, "time-pos") == 0) {
            double pos = *(double*)prop->data;
            for (int i=1; i<instances; i++) {
                mpv_set_property(player1, "time-pos", MPV_FORMAT_DOUBLE, &pos);
            }
        }
    }

    if (event->event_id == MPV_EVENT_SHUTDOWN)
        break;
}

`

Basically I subscribe to changes of the "time-pos" property, then set the time-pos of every slave player instance. This gets really close to what I want, but sometimes doubles a frame or skips a single frame and I'll end up off by one for a bit before returning to the correct timing. Is there a better method that I could use here? The time-pos property appears to update once per frame, so it is a bit course for this.

[ghost]

A better method would be adjusting playback speed for small deviations. Also "time-pos" is somewhat asynchronously reported, so it can definitely happen that it's off by a frame or more.

[bitbyt3r]

Ok, I can try that and see how it works out. I saw in issue #5592 that there may be a delay in adjusting the playback speed. I could probably work around that, but is that normal behavior?

I'm also working on code to sync the playback time across the network accounting for network delay and some amount of jitter. I'm writing all this as a separate executable that calls libmpv, but I could try to merge it into mpv itself. Would that be something you'd be interested in merging, or should I keep it separate?

Thanks for the help!

[hansinator]

Hi,
I need network sync support like in mplayer but with the ability sync the playlist position, i.e. the slaves should follow the master if the master plays another video in the playlist. I am currently using mplayer, but I would need to modify the mplayer network sync anyway (and make it incompatible to the upstream version), so I've decided it'd be a good opportunity to implement this whole thing in mpv. I am not convinced by a separate executable, plugin or script. I'd like to add support directly into mpv so that the feature is readily available via command line and for library users.

Do you have any suggestions or pointers into the right direction before I start? I have read that the blocking socket read behaviour on the slave side is undesired. Does anyone have an idea what a better solution might look like? However, I believe this is an implementation detail that can be changed at any time without affecting the network packet format or user experience. I've got a couple of days time at my disposal and a video wall with multiple slaves to test the whole thing.

Edit:
I've taken a closer look at the mpv code and did some tests with the existing udp_sync branch (rebased onto the current master). First I've measured the latency using a master and separate slave with identical hardware on a local network and a video that shows the played frame number without audio at 60FPS. Recording the playback with a high-speed camera shows that mplayer has a constant lag of about 1 frame. mpv has a lag of 1-2 frames. That's better than I expected from code using events and separate threads.

Now I could be satisfied, but I see a problem when the slave gets slightly out of sync. Mplayer has a pretty elegant solution to this using the blocking udp read. It is not as stupid as it might look first. The video playback loop essentially ceases to use it's own timing and relies on the operating system to block the thread until a UDP packet arrives. This ensures that the slave does not play the video faster then the master. It also has a very low latency, lower than propagating events via threads.
If the slave lags behind the master then UDP packets will queue up, so the code won't block and the slave playback loop iterates as fast as the UDP queue can be emptied while still decoding video frames. This will make the slave playback speed faster to catch up.
So essentially the way the network code is implemented (with all its ugly blocking ) automagically does the playback speed adjusting that wm4 suggested.

The question now is, how to implement that in mpv. I fear it'll be a lot more complex and messy when not using a blocking socket.

Edit2:
One thing about the playback speed adjustment that makes me uncomfortable is that when the video FPS matches the output FPS then faster playback is just dropping frames unless you can vary the output FPS quickly. Slowing down by blocking in turn will make one video frame be shown for multiple output frames and then just continue normally. It all results in stuttering motion which would be quite noticeable depending on the content.

Edit3:
@bitbyt3r have you had a look at PTP / IEEE1588 for syncing time while accounting for network delays? The linuxptp implementation has lots of code doing this with µs accuracy. One could use linuxptp to sync the time of multiple playback devices in a local network and then synchronized playback is as easy as the master transmitting the exact wall-clock time it plans to show the next frame. The slaves would just need to sleep until that moment and show the frame. Works well with software ptp for video applications, i.e. no phy/mac support of hardware timers necessary. The existing UDP sync with it's 1-2 frame latency however is more than good enough (given it does not seek) unless you have one continuous display surface where e.g. the left and right parts are connected to different playback devices, in which case frame-perfect sync is a necessity. In any case, it seems to me like implementing accounting for network delays yourself is overkill. If you don't want to sync the system clock, then ptp can just provide synced software/hardware timers. There is this phc2sys daemon which syncs the system clock to the hardware/software timers complete with servos for syncing gradually in small steps (no big step-change). It is a perfect starting point: in the code paths where it actually adjusts the system clock time, subtract an offset from the time and just call your mpv_set_property(player1, "time-pos", MPV_FORMAT_DOUBLE, &pos); instead and it is done on the slave side.

[nixternal]

I built mpv with the network sync stuff, but I am clueless on how to use it properly. Any help there will be greatly appreciated. Also, anyone done any further work since last year? mplayer is great and all, but when utilizing Intel graphics & libvdpau-va-gl anything other than h264 is a no go.

[Collocalini]

2nixternal:
use
{ "command": ["get_property", "speed"], "request_id": "asc" } to get playback speed
{ "command": ["get_property", "time-pos"], "request_id": "gtp" } to get current playback position

The request_id could be arbitrary.

{ "command": ["set_property", "speed", 1.0]} to set speed
{ "command": ["set_property", "time-pos", 0.0]} to set playback position

Commands should be sent through a socket. Run mpv like this:
mpv --input-ipc-server=/tmp/mpvsocket file.mp4

for more see section JSON IPC from https://mpv.io/manual/master/