Shaka-player: Multiple Video tile needs to seamless sync , test video is inside

[trustfarm-dev]

Have you read the FAQ and checked for duplicate open issues?:
yes
What version of Shaka Player are you using?:
latest
Can you reproduce the issue with our latest release version?:
yes
Can you reproduce the issue with the latest code from master?:
yes
Are you using the demo app or your own custom app?:
custom app
If custom app, can you reproduce the issue using our demo app?:
can not try in demo app.
What browser and OS are you using?:
windows10
What are the manifest and license server URIs?:

What did you do?

Following recommendation of issue #1752

We did sync with video.currentTime
TestCode is Here

And then every 1~2 seconds, try to sync works.
But, Buffering effects and shows redline border for shakaPlayer side.
TestVideo
Is there a way to remove buffering effects, and seamless sync way?
Need to any idea to suggestions.

Thanks.

[trustfarm-dev]

videoelem.currentTime = minTime;

in case of above setting , it will make force restart to sync process.
it makes artifact.
How can I call low-level (native time api) defered way , in shakaP ?

[trustfarm-dev]

issues #578 is a little bit help for sync not go out.
but, it is also happen frequently.

[TheModMaker]

First, setting the video's currentTime causes a seek. This can be an expensive operation, even if into a buffered region. I changed your code to avoid seeking if we are close to the time already and the buffering almost went entirely away.

Another tip is to pause all the videos until all of them have enough buffered. This ensures that one video buffering doesn't cause it to fall behind the others and avoids more seeks. While all of them are playing they should stay in sync, so you should only need to adjust if one of them starts buffering.

Also remember that you are downloading 16 different videos concurrently. This causes a lot of network traffic, so be sure to choose a low resolution. I have a really fast internet connection and it is almost continuously downloading media. Which is probably why it is buffering a lot. Our ABR logic assumes little or no external network traffic, so it won't work with multiple players; this means you should only have one resolution or manually switch between them.

Also note that the player has a buffering event that fires when we enter and leave a buffering state. You can use this to better provide the red border since using waiting and playing can be unstable and I see the border getting stuck on. The player also has an isBuffering method to check whether it is currently in a buffering state.

[joeyparrish]

@TheModMaker, I disagree that our ABR logic won't work with multiple players. I believe that estimating based on throughput with players constantly working will produce a reasonable estimate for what each can achieve.

@trustfarm-dev, I agree with @TheModMaker about the expense of seeking. Instead of seeking, I suggest adjusting video.playbackRate gently to get some tiles to catch up or slow down to sync with each other. For example, if one tile is behind the others, try setting playbackRate to 1.02. Or if one tile is too far ahead, it could have playbackRate of 0.99 to let the others catch up. I have seen some clever multi-device sync solutions that worked this way, though I don't know what algorithms they use or what their min/max playback rates are. The app would have to carefully coordinate them all.

It's also worth noting that players could get out of sync without a buffering event at all. Rendering of media in the browser or any other player is not so precise as you might expect. So long as it is accurate enough relative to human perception, it doesn't need to be perfect. For each individual player, this means sync between audio and video need to be close enough to look right to humans. But so long as that were true for each player, the browser would be under no obligation to keep multiple players in perfect sync with each other, even if their media were all 100% pre-buffered.

[hosung03]

@TheModMaker @joeyparrish

Thank you for your comments. Currently, our team are testing a single MPD. There are links per a bit rate: video.html information is here

So, we are planning to switch and provide an integrated mpd file for adaptation.
Now I know that "currentTime" is a function for "seek". I wonder how not to display the red line.
When adjusting the "playbackRate", it is difficult to find the point of time when the normal rate should be changed according to synchronizing. So, if you have a lite call of "currentTime", it would be better.

[trustfarm-dev]

@TheModMaker @joeyparrish @hosung03

shaka GapJumpingController code

stallPlayheadTime_ is it lite function ?

And, playhead seekRange related code will helpful?

[joeyparrish]

@hosung03, @trustfarm-dev, I don't understand some of your comments.

What do you mean when you say "how not to display the red line"? What red line is this? Shaka Player does not render the video. If there's a red line in the video, that's either something in the video content itself or some artifact of browser decoding.

What do you mean when you say "a lite call of currentTime"? I don't know what this means. video.currentTime is the API for seeking provided by the browser. It's part of HTML5 video.

@trustfarm-dev, I don't think gap jumping has anything to do with this. At least, I see no evidence of it.

[trustfarm-dev]

@joeyparrish Thanks for information.
We found that redline is marked in our app side,
and Video is in HTML5 MediaComponent, Not in shaka-players.

And, out side of shaka-player, I found that , fastseek in HTML5
https://html.spec.whatwg.org/multipage/media.html#dom-media-fastseek

Thanks again, close it.

[TheModMaker]

Just be careful, fastSeek() may not be precise. For example, it may move to a nearby keyframe instead of the time you passed in. If you are looking to keep the videos exactly synced, you'll need to do a normal seek. It is also only supported on Firefox, not on Chrome or Edge. See the MDN page for more info.