React-native-track-player: [iOS][1.1.4] App freezes when buffering a new track

[CapitanRedBeard]

Note This only occurs on the latest 1.1.4, 1.1.3 and below don't have this performance issue.

Problem

When playing an unbuffered track the app will completely freeze until the required buffering to play the app is completed. The UI thread will drop to close to 0, however no action can be taken (scroll, navigating, pressing anything) until the buffing is finished. This causes a dramatic performance issue whenever playing an audio.

By playing an unbuffered track I'm referring to these two scenarios:

1. Add N tracks on app load, then skip to one that is not the first.
   or
2. Reset the track player and add new tracks on some press handler in the app.

Video

react-native-track-player @ 1.1.4

You'll notice in this video it looks as if I'm pressing and holding the TouchableOpacity. That is not what's going on here, I'm just tapping it and it is freezing on it until the audio has finished buffered and then it is going back to what its doing. Same happens for any UI changes such as scrolling on a list.

react-native-track-player @ 1.1.3

Repro

I created a simplified repro here

The steps to repro are download the app:

git clone https://github.com/CapitanRedBeard/react-native-track-player-audio-bug-1.1.4.git
cd react-native-track-player-audio-bug-1.1.4
yarn
react-native link
yarn start

This will start the small repro demo with react-native-track-player @ 1.1.4
After playing with this remove the app, switch the version to 1.1.3 and restart the demo

You should notice that the TouchableOpacity buttons to play the audio no longer lag while the audio is buffing

Note This repro is extremely simplified, is a real use case scenario I would just queue up all those audio and use the skip API to navigate to the proper audio which I do in my app when I can. However there are a lot of places and scenarios where you simply need to reset the player and load up a new queue (Almost everywhere in my app in fact). So while this repro doesn't show the best use case of the library it is used to illustrate the bug that was introduced somewhere between 1.1.3 and 1.1.4. Also you can notice this bug in this very simple repro, but in a larger project with redux the problem is a lot more noticeable and the delay can be upwards of 2 seconds. The delay completely disallows the user from doing anything. For example is a scroll list you can't even scroll.

Possible Causes

I know that there was a lot of buffer work going on which I love but it seems that the audio and the frame rates don't pick up until the audio has finished buffering (I assume this because I print out when we're buffering and you see it clearly in 1.1.3 but not really in 1.1.4).

[CapitanRedBeard]

@dcvz I'm real bad on the iOS side so I'm having a hard time digging thru the code and finding what could of gone wrong but I'm 90% sure it has to be around the buffering work (which is great btw 😄) you've done recently because the frames seems to freeze until the buffering is done.
Pull down the repro, check it out, and let me know if there is anything else I can do to help narrow this bug down.

[curiousdustin]

While it may not be the standard use case, I DO make heavy use of reset. For a few reasons, I ended up managing my own queue, so I reset() -> add() each time I play a track.

Hopefully this is an easy fix.

[CapitanRedBeard]

@curiousdustin Yeah same with me it tends to allow a better queue management.

I just modifying my repro to add all the tracks initially and only skip around to tracks and the exact same bug still occurs. So its not specific to reseting and adding simply playing an unbuffered track will freeze the app until the buffering has finished.

Can be repro'd on the only-skip branch

Going to change the title as this is a more generic bug than I initially thought

[minhtc]

Try this commit: https://github.com/jorgenhenrichsen/SwiftAudio/pull/51

[thessler27]

@minhtc 's fixes applied to ios/vendor/AudioPlayer/AVPlayerWrapper/AVPlayerWrapper.swift fixes this bug

[fcaride]

Are you planning to merge this fix?

[thessler27]

@fcaride @Guichaguri @dcvz See #567

[dcvz]

Hey guys, sorry I've been a bit busy. I will look into it later today 🙏

[minhtc]

@thessler27 I think it should be fixed in Vendor module so I created pull request in jorgenhenrichsen/SwiftAudio instead here.
While module creators is busy, you can use patch-package to quick fix the issue.

[curiousdustin]

I had attempted a similar fix, but the asynchronous nature of the loading broke other functionality. For instance, I could no longer use:

await TrackPlayer.play();
await TrackPlayer.seekTo(x);

Because the play command would return before the current item was set and so the seek did not act on the expected track.

Does this fix have similar issues?

[minhtc]

@curiousdustin You need to listen ready state to perform seekTo function.

 TrackPlayer.addEventListener('playback-state', data => {
        if (data.state === TrackPlayer.STATE_READY) {
          TrackPlayer.seekTo(x)
        }
      })

I have patched a native solution to resolve seekTo right after async loading track, but not sure if anyone need it.

[curiousdustin]

Is this the way it has always been expected you would have to use seekTo()?

I have been using it without requiring the state to be STATE_READY, and it works fine.

Seems like there should be a way to not return the play() promise until the actual result of attempting to play occurs.

[donghungx]

dont know why i can't use 1.1.3 to get rid of this problem, it always download 1.1.4

[CapitanRedBeard]

@curiousdustin @minhtc Yeah I also have just been using the

await TrackPlayer.play();
await TrackPlayer.seekTo(x);

But if

TrackPlayer.addEventListener('playback-state', data => {
        if (data.state === TrackPlayer.STATE_READY) {
          TrackPlayer.seekTo(x)
        }
      })

is 100% reliable that sounds like a reasonable way to go about it. Honestly would be great if we could just do something like this TrackPlayer.play({seekTo: x}) or something to make it explicit

[curiousdustin]

On iOS, the SwiftAudio library being used already has the ability to set an initialTime for playback.

https://github.com/jorgenhenrichsen/SwiftAudio/commit/6cc0638a70af6d754a4a40cda03396fbea7981d9

This just needs to be hooked up to the custom track object used by RNTP.

I would submit a PR for this, however, I would also want it to work on Android (ExoPlayer), and I'm not sure of the best way to go about this.

In this solution, the "seek" would actually be pre determined in the track object. Alternatively, or additionally, it would be nice to have the possibility of doing something like @CapitanRedBeard suggested. TrackPlayer.play({seekTo: x}) or TrackPlayer.playFromTime(x)

Related:
https://github.com/react-native-kit/react-native-track-player/issues/387
https://github.com/react-native-kit/react-native-track-player/issues/496

[minhtc]

@curiousdustin @minhtc Yeah I also have just been using the

await TrackPlayer.play();
await TrackPlayer.seekTo(x);

But if

TrackPlayer.addEventListener('playback-state', data => {
        if (data.state === TrackPlayer.STATE_READY) {
          TrackPlayer.seekTo(x)
        }
      })

is 100% reliable that sounds like a reasonable way to go about it. Honestly would be great if we could just do something like this TrackPlayer.play({seekTo: x}) or something to make it explicit

Check out my full patch that support seekTo() right after play() : https://gist.github.com/minhtc/2586639a87e4a67d588da1736df7a800

[Guichaguri]

@curiousdustin Feel free to submit one, I can help you implementing it on Android.

[puckey]

I incorporated the PR from @minhtc because I was also suffering from the ios version freezing when buffering (in my case) radio streams. It works very well (aside from a minor issue: https://github.com/jorgenhenrichsen/SwiftAudio/pull/51/commits/4a512c6aa0e40a2f059781dfe9dd961d1322d0fb#r290260768) - thanks!

[curiousdustin]

@minhtc, I see that some of your changes were used in SwiftAudio. https://github.com/jorgenhenrichsen/SwiftAudio/pull/51

Are you using those changes in react-native-track-player?

Have you had any issues with lock screen / background behavior?

If you have a chance, can you look at the issues I'm having here https://github.com/react-native-kit/react-native-track-player/issues/709?