Newpipe: Release 0.18.0

[TobiGr]

New

• PeerTube support without P2P (#2201) [Beta]:

  • Watch and download videos from PeerTube instances
  • Add instances in the settings to access the complete PeerTube world
  • There might be problems with SSL handshakes on Android 4.4 and 7.1 when accessing certain instances resulting in a network error.

• Downloader (#2679):

  • Calculate download ETA
  • Download opus (webm files) as ogg
  • Recover expired download links to resume downloads after a long pause

• Adjustable seek duration #2744

Improved

• Make the KioskFragment aware of changes in the preferred content country and improve performance of all main tabs #2742
• Use new Localization and Downloader implementations from extractor #2713
• Make "Default kiosk" string translatable
• Black navigation bar for black theme #2569

Fixed

• Fixed a bug that could not move the popup player if another finger was placed while moving the popup player #2772
• Allow playlists missing an uploader and fix crashes related to this problem #2724, TeamNewPipe/NewPipeExtractor#219
• Enabling TLS1.1/1.2 on Android 4.4 devices (API 19/KitKat) to fix TLS handshake with MediaCCC and some PeerTube instances #2792
• [SoundCloud] Fixed client_id extraction TeamNewPipe/NewPipeExtractor#217

Development

• Update ExoPlayer to 2.10.8 #2791, #2816
• Update Gradle to 3.5.1 and add Kotlin support #2714

---

APK for testing: RC3

newpipe_debug_0.18.0_RC3.apk.zip

PLEASE REPORT ONLY _NEW_ ISSUES BELOW
Corresponding PR: #2862

[TobiGr]

This release contains wonderful changes. Unfortunately, I have other things to do atm and no time left for writing a release blog post. Can someone else do this? I'll provide help if needed.

Edit: I wrote a small post during a train ride.

[opusforlife2]

What do you mean by "release blog post"? Haven't you already covered everything above?

[TobiGr]

Not yet completely. In the blog post we mention the authors, sum up translation changes (e.g. New languages which have been added) and most importantly explain new features / UX. You cannot find everything in the few bullet points above :)

[TobiGr]

@thegodofplay I cannot reproduce. What content language/country are set on your device?

Edit: I uploaded RC2

[TobiGr]

Ah yes. That's for debugging :)

[hypergreatthing]

This doesn't seem to fix the soundcloud playback issue.

[B0pol]

PeerTube translations missing

New strings about PeerTube haven't been pushed to Weblate and then haven't been translated (french is 100% translated on Weblate).

[Stypox]

@B0pol This is normal, as I think translations are not pushed to Weblate until the update is released

[Stypox]

@yausername @TobiGr the text box dialog to add a new instance, under the "PeerTube instances" settings tab, is not correctly configured for these reasons:

• the text box expects normal text and not urls (so the keyboard tries to autocorrect)
• the left and right borders of the text box are too tight
• if there are spaces at the end of the URL the instance can't be validated (this could be solved by just stripping the string)

[TobiGr]

@yausername @TobiGr the text box dialog to add a new instance, under the "PeerTube instances" settings tab, is not correctly configured for these reasons:

@Stypox Good catch, could someone fix this? Should be quite easy.

This doesn't seem to fix the soundcloud playback issue.

@hypergreatthing Yes, that's right. There is still an auth problem when getting the stream links. Any kind of help is welcome.

Btw. I found time to write a blog post while sitting in a train. It's not perfect, but sufficient.

[eladkarako]

1. installing it over existing 17.4 cleared settings an db.
2. can not change language, changing it in settings does nothing. language displayed is device's default.

[yausername]

@Stypox Good catch, could someone fix this? Should be quite easy.

Fixed in https://github.com/TeamNewPipe/NewPipe/pull/2868

[TobiGr]

uploaded a third release client: newpipe_debug_0.18.0_RC3.apk.zip

installing it over existing 17.4 cleared settings an db.

@eladkarako This is a debug build, so signatures might differ from the version you installed previously. But most importantly, this is _not_ a release build. What 0.17.4 version did you install and where did it came from?

can not change language, changing it in settings does nothing. language displayed is device's default.

What did you try to do? Did you change the content country or content language? Both options do not change the app's display language.

[Nandan9]

Playing video as audio in the background of this 0.18.0_RC3.apk update causes scrolling problems.

[theScrabi]

Im having an issue that is also pressent in the current version of NewPipe, but also in the current debug build. When I try to open a stream on soundcloud i get an npe:

Exception

• __User Action:__ requested stream
• __Request:__ https://soundcloud.com/jedentageinset/mikah-7-jahre-jeden-tag-ein-set-harry-klein-munchen-18102019
• __Content Language:__ GB
• __Service:__ SoundCloud
• __Version:__ 0.18.0
• __OS:__ Linux Android 9 - 28

Crash log

org.schabi.newpipe.extractor.stream.StreamInfo$StreamExtractException: Could not get any stream. See error variable to get further details.
    at org.schabi.newpipe.extractor.stream.StreamInfo.extractStreams(StreamInfo.java:188)
    at org.schabi.newpipe.extractor.stream.StreamInfo.getInfo(StreamInfo.java:68)
    at org.schabi.newpipe.extractor.stream.StreamInfo.getInfo(StreamInfo.java:60)
    at org.schabi.newpipe.util.ExtractorHelper.lambda$getStreamInfo$3(ExtractorHelper.java:115)
    at org.schabi.newpipe.util.-$$Lambda$ExtractorHelper$5fJcha6Sq5APJBLdG6osaJby-mc.call(Unknown Source:4)
    at io.reactivex.internal.operators.single.SingleFromCallable.subscribeActual(SingleFromCallable.java:44)
    at io.reactivex.Single.subscribe(Single.java:3438)
    at io.reactivex.internal.operators.single.SingleDoOnSuccess.subscribeActual(SingleDoOnSuccess.java:35)
    at io.reactivex.Single.subscribe(Single.java:3438)
    at io.reactivex.internal.operators.maybe.MaybeFromSingle.subscribeActual(MaybeFromSingle.java:41)
    at io.reactivex.Maybe.subscribe(Maybe.java:4154)
    at io.reactivex.internal.operators.maybe.MaybeConcatArray$ConcatMaybeObserver.drain(MaybeConcatArray.java:153)
    at io.reactivex.internal.operators.maybe.MaybeConcatArray$ConcatMaybeObserver.request(MaybeConcatArray.java:78)
    at io.reactivex.internal.operators.flowable.FlowableElementAtMaybe$ElementAtSubscriber.onSubscribe(FlowableElementAtMaybe.java:66)
    at io.reactivex.internal.operators.maybe.MaybeConcatArray.subscribeActual(MaybeConcatArray.java:42)
    at io.reactivex.Flowable.subscribe(Flowable.java:14479)
    at io.reactivex.internal.operators.flowable.FlowableElementAtMaybe.subscribeActual(FlowableElementAtMaybe.java:36)
    at io.reactivex.Maybe.subscribe(Maybe.java:4154)
    at io.reactivex.internal.operators.maybe.MaybeToSingle.subscribeActual(MaybeToSingle.java:46)
    at io.reactivex.Single.subscribe(Single.java:3438)
    at io.reactivex.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.run(SingleSubscribeOn.java:89)
    at io.reactivex.Scheduler$DisposeTask.run(Scheduler.java:578)
    at io.reactivex.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:66)
    at io.reactivex.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java:57)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:301)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
    at java.lang.Thread.run(Thread.java:764)

---

[TobiGr]

@theScrabi https://github.com/TeamNewPipe/NewPipeExtractor/pull/217#issuecomment-564038173
I was able to get the client_id extraction working again. But for some reason, accessing stream data results in a 401 response

[eladkarako]

This is a debug build, so signatures might differ

@TobiGr then scratch that there are no bugs then.

_{I've used production-version of 17.4, so it uses a different signature which means it works side by side with the 18.x-debug version.
as for the language, it seems that there were new language-resources added,
one of them is my native-phone's, which means that Android prefers it over English (I preferred English),
that isn't an issue I'll use APKTool to delete all the language resources but English and recompile the APK, I really hope there would be a language selection option, or at least a checkbox to "force English" like several other applications. https://github.com/TeamNewPipe/NewPipe/issues/2162}

[theScrabi]

You can submit this as a new feature request.

[irredentist]

Just wondering, but is there supposed to be support for playing Peertube videos in the background?

[TobiGr]

Regarding SoundClound: I was able to extract the correct stream urls, but I am still facing problems (see my https://github.com/TeamNewPipe/NewPipe/issues/2823#issuecomment-567314387). I'd suggest to release 0.18.0 and a hotfix asap. @theScrabi what do you think?

[ADepic]

Is there any work on sorting feed by date? Currently using apks from https://github.com/TeamNewPipe/NewPipe/pull/2547 because it sorts the feed chronologically.

[sliceanddice9]

Will this newly updated 0.18.0 version of Newpipe fix the "video is unavailable" problem? It seemsthat most videos have this message for me now, this wasn't happening before. Is it happening to anyone else?

[yausername]

@TobiGr @theScrabi
On what environment are you generating the apk?
If we generate the apk on linux preferably debian/ubuntu with openjdk-8-jdk, we can easily match the reproducibility criteria of fdroid. That way it will be possible for us to publish apk signed by @theScrabi on fdroid.

[TobiGr]

@yausername That's a good hint! Thanks. I've used the JDK which is embedded in Android Studio.

Will this newly updated 0.18.0 version of Newpipe fix the "video is unavailable" problem? It seemsthat most videos have this message for me now, this wasn't happening before. Is it happening to anyone else?

I was not able to repo this problem. We need to ship the extended error report to get more info on issues like this one. I'll try to include it in the next release.

Just wondering, but is there supposed to be support for playing Peertube videos in the background?

I could not find an audio-only stream on PeerTube when I reviewed the PR. However, there is a PR, which was merged recently, that might be interesting to get the required info: https://github.com/Chocobozzz/PeerTube/pull/2213 CC @yausername :) @sliceanddice9 Can you open an issue so we do not forget about this?

Is there any work on sorting feed by date? Currently using apks from #2547 because it sorts the feed chronologically.

@ADepic Yes, there are two PRs which aim to fix that problem. I did not have time to review both, but that is right at the top of my todo list as soon as 0.18.0 is released.

[TobiGr]

@yausername I used both jdks and got two apks with the same sha256 sum (and cleaned the project between both builds).

[TobiGr]

SoundCloud will be fixed in the update 🎉

[yausername]

@yausername I used both jdks and got two apks with the same sha256 sum (and cleaned the project between both builds).

Can you share the apk you built and the commit id from which it was built? I will build myself and compare the two using https://github.com/signalapp/Signal-Android/blob/master/apkdiff/apkdiff.py
This is not quite the way fdroid compares them but it should pass nonetheless.

FYI I tried to compare the RC3 apk here to the one I built from release_0.18.0 and they did not match.

[theScrabi]

@yausername if its reproducable build you tried: That is complicated. You can build the same code twice on the samr machine and it may produce different apks.

[yausername]

@yausername if its reproducable build you tried: That is complicated. You can build the same code twice on the samr machine and it may produce different apks.

I don't see this behavior at all. For example I am able to reproducibly build the v0.17.4 apk on my local machine which verifies using fdroid verify org.schabi.newpipe command

[snappyapple632]

This crash occurs when I try to share a downloaded audio file using the "Share" button:

Exception

• __User Action:__ ui error
• __Request:__ App crash, UI failure
• __Content Language:__ US
• __Service:__ none
• __Version:__ 0.18.0 RC3
• __OS:__ Linux Android 10 - 29

Crash log

```
android.os.FileUriExposedException: file:///storage/emulated/0/Music/NewPipe/Hold%20the%20Line.m4a exposed beyond app through ClipData.Item.getUri()
at android.os.StrictMode.onFileUriExposed(StrictMode.java:2083)
at android.net.Uri.checkFileUriExposed(Uri.java:2388)
at android.content.ClipData.prepareToLeaveProcess(ClipData.java:977)
at android.content.Intent.prepareToLeaveProcess(Intent.java:10771)
at android.content.Intent.prepareToLeaveProcess(Intent.java:10777)
at android.content.Intent.prepareToLeaveProcess(Intent.java:10756)
at android.app.Instrumentation.execStartActivity(Instrumentation.java:1889)
at android.app.Activity.startActivityForResult(Activity.java:5205)
at androidx.fragment.app.FragmentActivity.startActivityForResult(FragmentActi

[snappyapple632]

Update: The first error might have been due to OnePlus's File Manager not allowing access to the share function until I agreed to their terms. Now I get this error:

Exception

• __User Action:__ ui error
• __Request:__ App crash, UI failure
• __Content Language:__ US
• __Service:__ none
• __Version:__ 0.18.0 RC3
• __OS:__ Linux Android 10 - 29

Crash log

android.os.FileUriExposedException: file:///storage/emulated/0/Music/NewPipe/Hold%20the%20Line.m4a exposed beyond app through ClipData.Item.getUri()
    at android.os.StrictMode.onFileUriExposed(StrictMode.java:2083)
    at android.net.Uri.checkFileUriExposed(Uri.java:2388)
    at android.content.ClipData.prepareToLeaveProcess(ClipData.java:977)
    at android.content.Intent.prepareToLeaveProcess(Intent.java:10771)
    at android.content.Intent.prepareToLeaveProcess(Intent.java:10777)
    at android.content.Intent.prepareToLeaveProcess(Intent.java:10756)
    at android.app.Instrumentation.execStartActivity(Instrumentation.java:1889)
    at android.app.Activity.startActivityForResult(Activity.java:5205)
    at androidx.fragment.app.FragmentActivity.startActivityForResult(FragmentActivity.java:676)
    at android.app.Activity.startActivityForResult(Activity.java:5163)
    at androidx.fragment.app.FragmentActivity.startActivityForResult(FragmentActivity.java:663)
    at android.app.Activity.startActivity(Activity.java:5534)
    at android.app.Activity.startActivity(Activity.java:5502)
    at us.shandian.giga.ui.adapter.MissionAdapter.shareFile(MissionAdapter.java:372)
    at us.shandian.giga.ui.adapter.MissionAdapter.handlePopupItem(MissionAdapter.java:608)
    at us.shandian.giga.ui.adapter.MissionAdapter.access$200(MissionAdapter.java:81)
    at us.shandian.giga.ui.adapter.MissionAdapter$ViewHolderItem.lambda$buildPopup$3$MissionAdapter$ViewHolderItem(MissionAdapter.java:881)
    at us.shandian.giga.ui.adapter.-$$Lambda$MissionAdapter$ViewHolderItem$RgJD8t6gH83XiFLVb3LGB9-h96A.onMenuItemClick(Unknown Source:2)
    at android.widget.PopupMenu$1.onMenuItemSelected(PopupMenu.java:108)
    at com.android.internal.view.menu.MenuBuilder.dispatchMenuItemSelected(MenuBuilder.java:787)
    at com.android.internal.view.menu.MenuItemImpl.invoke(MenuItemImpl.java:151)
    at com.android.internal.view.menu.MenuBuilder.performItemAction(MenuBuilder.java:934)
    at com.android.internal.view.menu.MenuBuilder.performItemAction(MenuBuilder.java:924)
    at com.android.internal.view.menu.MenuPopup.onItemClick(MenuPopup.java:128)
    at android.widget.AdapterView.performItemClick(AdapterView.java:330)
    at android.widget.AbsListView.performItemClick(AbsListView.java:1257)
    at android.widget.AbsListView$PerformClick.run(AbsListView.java:3265)
    at android.os.Handler.handleCallback(Handler.java:883)
    at android.os.Handler.dispatchMessage(Handler.java:100)
    at android.os.Looper.loop(Looper.java:214)
    at android.app.ActivityThread.main(ActivityThread.java:7682)
    at java.lang.reflect.Method.invoke(Native Method)
    at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:516)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:950)

---

[TobiGr]

@theScrabi Can you release the new version? I asked @TheAssassin to sign the apk before i managed to fix soundcloud which he did. But I didn't get the signed apk containing the sc fixes from him yet.

[yausername]

@theScrabi @TobiGr just to add that v0.17.4 of newpipe is already reproducible according to fdroid. https://verification.f-droid.org/verified.json (search for org.schabi.newpipe).

We can publish apk signed by @theScrabi just by adding signatures to the fdroid metadata, similar to https://gitlab.com/fdroid/fdroiddata/tree/master/metadata/de.schildbach.wallet/signatures

[ksio89]

Thanks for the new build, but videos in WebM and 3GP containers are still not seekable, as I've reported in #2676. Also the video duration is not displayed on the thumbnail (I have MX Player installed), most likely the issues are related.