NewPipe doesn't properly shut down audio streams when playback is finished/interrupted

[Yowlen]

• [X] I carefully read the contribution guidelines and agree to them.
• [X] I checked if the issue/feature exists in the latest version.
• [ ] I did use the incredible bugreport to markdown converter to paste bug reports.

NewPipe ends up leaving open the streams when it's done using them. Even worse, NewPipe doesn't recycle old streams, causing a memory leak to form in Android's internal mediaserver over time, as it requires explicit commands to be given in order to free memory used by audio it plays. On low-RAM devices, this can cause problems if the user starts playing multiple videos.

Basically, the way I found this out is via an equalizer app that I use due to my sensitivity with certain frequencies:
https://play.google.com/store/apps/details?id=com.devdnua.equalizer.free

In this app, I was exploring the settings to make sure everything was just the way I want them, and I found the "View Active Audio Sessions" area. NewPipe had a ton in this list, despite no actively-playing videos at the time, so I began to investigate. The summary of my findings with NewPipe is as follows:

• If nothing is currently playing, and the user starts a new video, NewPipe will open a new stream. This includes having a video/audio stream currently open, but stopped/paused.
• If there's currently something playing and another of the same type is enqueued, NewPipe will use the existing stream when the new one starts.
• If there's currently something playing and the user replaces it with another of the same type, a new stream is started.
• If audio is playing and the user starts up a video, NewPipe starts a new stream. Same goes for if video is playing and the user starts an audio stream.
• Switching between pop-up and full screen recycles the old stream.
• At no point does NewPipe actually close a given stream.
  
  
  
  • This includes when the user manually closes a stream, regardless of whether the stream is still playing or not.
  
  

I should note a few other things, as well:

• The equalizer app can't actually close the streams itself, as they are owned by NewPipe and the system's equalizer abilities only allow for a passthrough adjustment for apps like this, not a complete change of ownership. Using the "X" button in the active streams window only tells the equalizer to stop filtering. As a result, the memory leak will still occur even after I remove a stream from the list.
• Force Stopping NewPipe also does not get rid of the memory leak. I assume this is to allow for apps to resume using a given stream even after such a close. I've seen other apps such as BubbleUPNP and Vanilla Music do this.
• The only two ways available to users to get rid of the memory leak here are as follows:
  
  
  
  • Reboot the device.
  
  
  • Rooted users can alternatively kill the mediaserver by running killall -9 mediaserver as root in a terminal or something. This will kill all streams, including currently-playing ones, so if NewPipe is currently playing something and the terminal command is used, the audio will cut out and NewPipe will never be the wiser. Video will still play. However, it will be silent until NewPipe starts a new stream via one of the methods outlined above.
  
  

The recommended solution is to have NewPipe close out each stream when it's done playing, as well as have it recycle streams when shutting down currently-playing items in favor of new ones. Also, audio and video currently use separate streams, so replacing them so that only one notification/stream or the other can exist at one time would also be helpful.

[goxr3plus]

Very interesting that should for sure be resolved.

[Redirion]

ExoPlayer 2.10. mitigates this issue. Not by recycling, but by reusing.

[ackzsel]

Could this also explain why my headset's play button doesn't operate my music player anymore after playing a video? It starts an empty 'unknown' stream instead until I force music playback from the music player.

[Yowlen]

Very possible. If your headset doesn't actively check to see which of the active streams are actually playing stuff and instead just takes the first available stream, it would likely lock itself into NewPipe's dead stream until you tell it to change.

[Yowlen]

I just want to remind people that this is still a major problem, even in the latest release. Watching around half an hour's worth of videos in NewPipe can easily eat up 300MB of RAM in my own testing.

And due to mediaserver's inability to cache active streams, as well as Android's own ZSWAP configuration that automatically uses up half the available RAM for swap space, this makes this a critical issue on devices with 2GB or less of total RAM. 30-60 minutes of videos can destroy a user's ability to run other apps on 1GB devices until rebooting, and 1-2 hours can do it for 2GB devices.

[Stypox]

Related: #2257

[Stypox]

Issue #3425 from @ravilov

So I've had this issue for a long time where after using NP for a while and watching some number of videos NP would suddenly not load videos anymore. It would show the video page, it would load the comments, but in the video box on top it would just show the spinner and would sit like that infinitely. From that point on it would refuse to load any other videos as well, in exactly the same way (so it's not some weird issue with just that one video). The only solution would be to force-stop NP and restart. It would then work for a while and then refuse to load again.

This issue happens so randomly and is quite hard to reproduce consistently so I just learned to live with it, restarting NP as necessary. Today though I found this in my logcat shortly after it happened again:

E dalvikvm-heap: Out of memory on a 2570616-byte allocation.
I dalvikvm: "RxCachedThreadScheduler-44" daemon prio=5 tid=15 RUNNABLE
I dalvikvm:   | group="main" sCount=0 dsCount=0 obj=0x433f9a90 self=0x61fc85f8
I dalvikvm:   | sysTid=6575 nice=0 sched=0/0 cgrp=apps handle=1643731624
I dalvikvm:   | state=R schedstat=( 1911832709 174470538 2801 ) utm=175 stm=15 core=3
I dalvikvm:   at java.lang.String.<init>(String.java:~422)
I dalvikvm:   at java.lang.AbstractStringBuilder.toString(AbstractStringBuilder.java:642)
I dalvikvm:   at java.lang.StringBuilder.toString(StringBuilder.java:663)
I dalvikvm:   at java.lang.String.replace(String.java:1396)
I dalvikvm:   at org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeStreamExtractor.loadDecryptionCode(YoutubeStreamExtractor.java:750)
I dalvikvm:   at org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeStreamExtractor.onFetchPage(YoutubeStreamExtractor.java:643)
I dalvikvm:   at org.schabi.newpipe.extractor.Extractor.fetchPage(Extractor.java:56)
I dalvikvm:   at org.schabi.newpipe.extractor.stream.StreamInfo.getInfo(StreamInfo.java:65)
I dalvikvm:   at org.schabi.newpipe.extractor.stream.StreamInfo.getInfo(StreamInfo.java:61)
I dalvikvm:   at org.schabi.newpipe.util.ExtractorHelper.lambda$getStreamInfo$3(ExtractorHelper.java:120)
I dalvikvm:   at org.schabi.newpipe.util.-$$Lambda$ExtractorHelper$5fJcha6Sq5APJBLdG6osaJby-mc.call(lambda:-1)
I dalvikvm:   at io.reactivex.internal.operators.single.SingleFromCallable.subscribeActual(SingleFromCallable.java:44)
I dalvikvm:   at io.reactivex.Single.subscribe(Single.java:3438)
I dalvikvm:   at io.reactivex.internal.operators.single.SingleDoOnSuccess.subscribeActual(SingleDoOnSuccess.java:35)
I dalvikvm:   at io.reactivex.Single.subscribe(Single.java:3438)
I dalvikvm:   at io.reactivex.internal.operators.maybe.MaybeFromSingle.subscribeActual(MaybeFromSingle.java:41)
I dalvikvm:   at io.reactivex.Maybe.subscribe(Maybe.java:4154)
I dalvikvm:   at io.reactivex.internal.operators.maybe.MaybeConcatArray$ConcatMaybeObserver.drain(MaybeConcatArray.java:153)
I dalvikvm:   at io.reactivex.internal.operators.maybe.MaybeConcatArray$ConcatMaybeObserver.request(MaybeConcatArray.java:78)
I dalvikvm:   at io.reactivex.internal.operators.flowable.FlowableElementAtMaybe$ElementAtSubscriber.onSubscribe(FlowableElementAtMaybe.java:66)
I dalvikvm:   at io.reactivex.internal.operators.maybe.MaybeConcatArray.subscribeActual(MaybeConcatArray.java:42)
I dalvikvm:   at io.reactivex.Flowable.subscribe(Flowable.java:14479)
I dalvikvm:   at io.reactivex.internal.operators.flowable.FlowableElementAtMaybe.subscribeActual(FlowableElementAtMaybe.java:36)
I dalvikvm:   at io.reactivex.Maybe.subscribe(Maybe.java:4154)
I dalvikvm:   at io.reactivex.internal.operators.maybe.MaybeToSingle.subscribeActual(MaybeToSingle.java:46)
I dalvikvm:   at io.reactivex.Single.subscribe(Single.java:3438)
I dalvikvm:   at io.reactivex.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.run(SingleSubscribeOn.java:89)
I dalvikvm:   at io.reactivex.Scheduler$DisposeTask.run(Scheduler.java:578)
I dalvikvm:   at io.reactivex.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:66)
I dalvikvm:   at io.reactivex.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java:57)
I dalvikvm:   at java.util.concurrent.FutureTask.run(FutureTask.java:237)
I dalvikvm:   at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:152)
I dalvikvm:   at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:265)
I dalvikvm:   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
I dalvikvm:   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
I dalvikvm:   at java.lang.Thread.run(Thread.java:841)
I dalvikvm: 
E class org.schabi.newpipe.App: RxJavaPlugins.ErrorHandler called with -> : throwable = [io.reactivex.exceptions.UndeliverableException]
E class org.schabi.newpipe.App: RxJavaPlugin: Undeliverable Exception received: 
E class org.schabi.newpipe.App: java.lang.OutOfMemoryError
E class org.schabi.newpipe.App:     at java.lang.String.<init>(String.java:422)
E class org.schabi.newpipe.App:     at java.lang.AbstractStringBuilder.toString(AbstractStringBuilder.java:642)
E class org.schabi.newpipe.App:     at java.lang.StringBuilder.toString(StringBuilder.java:663)
E class org.schabi.newpipe.App:     at java.lang.String.replace(String.java:1396)
E class org.schabi.newpipe.App:     at org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeStreamExtractor.loadDecryptionCode(YoutubeStreamExtractor.java:750)
E class org.schabi.newpipe.App:     at org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeStreamExtractor.onFetchPage(YoutubeStreamExtractor.java:643)
E class org.schabi.newpipe.App:     at org.schabi.newpipe.extractor.Extractor.fetchPage(Extractor.java:56)
E class org.schabi.newpipe.App:     at org.schabi.newpipe.extractor.stream.StreamInfo.getInfo(StreamInfo.java:65)
E class org.schabi.newpipe.App:     at org.schabi.newpipe.extractor.stream.StreamInfo.getInfo(StreamInfo.java:61)
E class org.schabi.newpipe.App:     at org.schabi.newpipe.util.ExtractorHelper.lambda$getStreamInfo$3(ExtractorHelper.java:120)
E class org.schabi.newpipe.App:     at org.schabi.newpipe.util.-$$Lambda$ExtractorHelper$5fJcha6Sq5APJBLdG6osaJby-mc.call(lambda)
E class org.schabi.newpipe.App:     at io.reactivex.internal.operators.single.SingleFromCallable.subscribeActual(SingleFromCallable.java:44)
E class org.schabi.newpipe.App:     at io.reactivex.Single.subscribe(Single.java:3438)
E class org.schabi.newpipe.App:     at io.reactivex.internal.operators.single.SingleDoOnSuccess.subscribeActual(SingleDoOnSuccess.java:35)
E class org.schabi.newpipe.App:     at io.reactivex.Single.subscribe(Single.java:3438)
E class org.schabi.newpipe.App:     at io.reactivex.internal.operators.maybe.MaybeFromSingle.subscribeActual(MaybeFromSingle.java:41)
E class org.schabi.newpipe.App:     at io.reactivex.Maybe.subscribe(Maybe.java:4154)
E class org.schabi.newpipe.App:     at io.reactivex.internal.operators.maybe.MaybeConcatArray$ConcatMaybeObserver.drain(MaybeConcatArray.java:153)
E class org.schabi.newpipe.App:     at io.reactivex.internal.operators.maybe.MaybeConcatArray$ConcatMaybeObserver.request(MaybeConcatArray.java:78)
E class org.schabi.newpipe.App:     at io.reactivex.internal.operators.flowable.FlowableElementAtMaybe$ElementAtSubscriber.onSubscribe(FlowableElementAtMaybe.java:66)
E class org.schabi.newpipe.App:     at io.reactivex.internal.operators.maybe.MaybeConcatArray.subscribeActual(MaybeConcatArray.java:42)
E class org.schabi.newpipe.App:     at io.reactivex.Flowable.subscribe(Flowable.java:14479)
E class org.schabi.newpipe.App:     at io.reactivex.internal.operators.flowable.FlowableElementAtMaybe.subscribeActual(FlowableElementAtMaybe.java:36)
E class org.schabi.newpipe.App:     at io.reactivex.Maybe.subscribe(Maybe.java:4154)
E class org.schabi.newpipe.App:     at io.reactivex.internal.operators.maybe.MaybeToSingle.subscribeActual(MaybeToSingle.java:46)
E class org.schabi.newpipe.App:     at io.reactivex.Single.subscribe(Single.java:3438)
E class org.schabi.newpipe.App:     at io.reactivex.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.run(SingleSubscribeOn.java:89)
E class org.schabi.newpipe.App:     at io.reactivex.Scheduler$DisposeTask.run(Scheduler.java:578)
E class org.schabi.newpipe.App:     at io.reactivex.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:66)
E class org.schabi.newpipe.App:     at io.reactivex.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java:57)
E class org.schabi.newpipe.App:     at java.util.concurrent.FutureTask.run(FutureTask.java:237)
E class org.schabi.newpipe.App:     at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:152)
E class org.schabi.newpipe.App:     at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:265)
E class org.schabi.newpipe.App:     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
E class org.schabi.newpipe.App:     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
E class org.schabi.newpipe.App:     at java.lang.Thread.run(Thread.java:841)

I'm hoping this might help eliminate this issue for good. Fingers crossed.

[gappleto97]

Is it possible that this issue is also what causes NewPipe to start an empty audio playlist if a Bluetooth device sends the play command after NewPipe has been closed?

[teemue]

What's the status of this issue?

[Yowlen]

Still an issue as of v0.19.5 (the latest on F-Droid atm)

Edit: Just want to add that at least in my opinion, it should ideally reuse the same stream for the entire playlist until playback is ended (which is already does), as well as for any subsequent streams played while that pop-up window is still open (which it doesn't) before closing out the stream when the window, pop-up or otherwise, is closed (which it also doesn't do).

Similarly, if the user pauses playback, the stream should be remembered and reused if a new item is begun before the paused stream is closed out.

And lastly, maybe merge audio-only and video playbacks into a single list and notification? Otherwise we'll run into complications with multiple open streams at once since it would require implementing a mixer inside NewPipe to keep things separate while still using the same mediaplayer stream.

[opusforlife2]

@Yowlen Could you test 0.20.0?

[Yowlen]

Ah, yeah. Sorry. I tested when I first saw the update to the new system, but forgot to post here. It's still happening. On the plus side, switching between fullscreen and the portrait view player doesn't create any new instances, but stopping playback still leaves behind the mediaserver instance.

The screenshot here is my equalizer using 0.20.0 after testing both the regular player, then restarting the video in pop-up mode and stopping it. NewPipe is completely closed at this point.

[opusforlife2]

@Redirion What say? Exoplayer issue?