Newpipe: Cannot Decrypt URL error since today

[ekianjo]

As of the 14th of March it looks like newpipe 0.16 cannot play videos anymore. I use it everyday and noticed this morning.

[AnotherLife]

Yes, me too.

Exception

• __User Action:__ requested stream
• __Request:__ https://www.youtube.com/watch?v=gQHBvxlf12Y
• __Content Language:__ GB
• __Service:__ YouTube
• __Version:__ 0.16.0
• __OS:__ Linux Android 4.4.4 - 19

Crash log

org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeStreamExtractor$DecryptException: Could not parse decrypt function 
    at org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeStreamExtractor.loadDecryptionCode(YoutubeStreamExtractor.java:739)
    at org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeStreamExtractor.onFetchPage(YoutubeStreamExtractor.java:614)
    at org.schabi.newpipe.extractor.Extractor.fetchPage(Extractor.java:52)
    at org.schabi.newpipe.extractor.stream.StreamInfo.getInfo(StreamInfo.java:63)
    at org.schabi.newpipe.extractor.stream.StreamInfo.getInfo(StreamInfo.java:59)
    at org.schabi.newpipe.util.ExtractorHelper.lambda$getStreamInfo$3(ExtractorHelper.java:115)
    at org.schabi.newpipe.util.-$$Lambda$ExtractorHelper$5fJcha6Sq5APJBLdG6osaJby-mc.call(lambda)
    at io.reactivex.internal.operators.single.SingleFromCallable.subscribeActual(SingleFromCallable.java:44)
    at io.reactivex.Single.subscribe(Single.java:3438)
    at io.reactivex.internal.operators.single.SingleDoOnSuccess.subscribeActual(SingleDoOnSuccess.java:35)
    at io.reactivex.Single.subscribe(Single.java:3438)
    at io.reactivex.internal.operators.maybe.MaybeFromSingle.subscribeActual(MaybeFromSingle.java:41)
    at io.reactivex.Maybe.subscribe(Maybe.java:4154)
    at io.reactivex.internal.operators.maybe.MaybeConcatArray$ConcatMaybeObserver.drain(MaybeConcatArray.java:153)
    at io.reactivex.internal.operators.maybe.MaybeConcatArray$ConcatMaybeObserver.request(MaybeConcatArray.java:78)
    at io.reactivex.internal.operators.flowable.FlowableElementAtMaybe$ElementAtSubscriber.onSubscribe(FlowableElementAtMaybe.java:66)
    at io.reactivex.internal.operators.maybe.MaybeConcatArray.subscribeActual(MaybeConcatArray.java:42)
    at io.reactivex.Flowable.subscribe(Flowable.java:14479)
    at io.reactivex.internal.operators.flowable.FlowableElementAtMaybe.subscribeActual(FlowableElementAtMaybe.java:36)
    at io.reactivex.Maybe.subscribe(Maybe.java:4154)
    at io.reactivex.internal.operators.maybe.MaybeToSingle.subscribeActual(MaybeToSingle.java:46)
    at io.reactivex.Single.subscribe(Single.java:3438)
    at io.reactivex.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.run(SingleSubscribeOn.java:89)
    at io.reactivex.Scheduler$DisposeTask.run(Scheduler.java:578)
    at io.reactivex.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:66)
    at io.reactivex.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java:57)
    at java.util.concurrent.FutureTask.run(FutureTask.java:237)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:152)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:265)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
    at java.lang.Thread.run(Thread.java:841)
Caused by: org.schabi.newpipe.extractor.utils.Parser$RegexException: failed to find pattern ";([A-Za-z0-9_\$]{2})\...\( inside of var u=function(a){return"string"==typeof a};"
    at org.schabi.newpipe.extractor.utils.Parser.matchGroup(Parser.java:73)
    at org.schabi.newpipe.extractor.utils.Parser.matchGroup(Parser.java:61)
    at org.schabi.newpipe.extractor.utils.Parser.matchGroup1(Parser.java:52)
    at org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeStreamExtractor.loadDecryptionCode(YoutubeStreamExtractor.java:726)
    ... 31 more

---

[boardfish]

Also experienced this just now.

{
   "user_action": "requested stream",
   "request": "https:\/\/www.youtube.com\/watch?v=RepXOI13fVI",
   "content_language": "GB",
   "service": "YouTube",
   "package": "org.schabi.newpipe",
   "version": "0.16.0",
   "os": "Linux Android 8.1.0 - 27",
   "time": "2019-03-14 00:04",
   "exceptions": [
      "org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeStreamExtractor$DecryptException: Could not parse decrypt function \n\tat org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeStreamExtractor.loadDecryptionCode(YoutubeStreamExtractor.java:739)\n\tat org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeStreamExtractor.onFetchPage(YoutubeStreamExtractor.java:614)\n\tat org.schabi.newpipe.extractor.Extractor.fetchPage(Extractor.java:52)\n\tat org.schabi.newpipe.extractor.stream.StreamInfo.getInfo(StreamInfo.java:63)\n\tat org.schabi.newpipe.extractor.stream.StreamInfo.getInfo(StreamInfo.java:59)\n\tat org.schabi.newpipe.util.ExtractorHelper.lambda$getStreamInfo$3(ExtractorHelper.java:115)\n\tat org.schabi.newpipe.util.-$$Lambda$ExtractorHelper$5fJcha6Sq5APJBLdG6osaJby-mc.call(Unknown Source:4)\n\tat io.reactivex.internal.operators.single.SingleFromCallable.subscribeActual(SingleFromCallable.java:44)\n\tat io.reactivex.Single.subscribe(Single.java:3438)\n\tat io.reactivex.internal.operators.single.SingleDoOnSuccess.subscribeActual(SingleDoOnSuccess.java:35)\n\tat io.reactivex.Single.subscribe(Single.java:3438)\n\tat io.reactivex.internal.operators.maybe.MaybeFromSingle.subscribeActual(MaybeFromSingle.java:41)\n\tat io.reactivex.Maybe.subscribe(Maybe.java:4154)\n\tat io.reactivex.internal.operators.maybe.MaybeConcatArray$ConcatMaybeObserver.drain(MaybeConcatArray.java:153)\n\tat io.reactivex.internal.operators.maybe.MaybeConcatArray$ConcatMaybeObserver.request(MaybeConcatArray.java:78)\n\tat io.reactivex.internal.operators.flowable.FlowableElementAtMaybe$ElementAtSubscriber.onSubscribe(FlowableElementAtMaybe.java:66)\n\tat io.reactivex.internal.operators.maybe.MaybeConcatArray.subscribeActual(MaybeConcatArray.java:42)\n\tat io.reactivex.Flowable.subscribe(Flowable.java:14479)\n\tat io.reactivex.internal.operators.flowable.FlowableElementAtMaybe.subscribeActual(FlowableElementAtMaybe.java:36)\n\tat io.reactivex.Maybe.subscribe(Maybe.java:4154)\n\tat io.reactivex.internal.operators.maybe.MaybeToSingle.subscribeActual(MaybeToSingle.java:46)\n\tat io.reactivex.Single.subscribe(Single.java:3438)\n\tat io.reactivex.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.run(SingleSubscribeOn.java:89)\n\tat io.reactivex.Scheduler$DisposeTask.run(Scheduler.java:578)\n\tat io.reactivex.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:66)\n\tat io.reactivex.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java:57)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:301)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1162)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:636)\n\tat java.lang.Thread.run(Thread.java:764)\nCaused by: org.schabi.newpipe.extractor.utils.Parser$RegexException: failed to find pattern \";([A-Za-z0-9_\\$]{2})\\...\\( inside of var u=function(a){return\"string\"==typeof a};\"\n\tat org.schabi.newpipe.extractor.utils.Parser.matchGroup(Parser.java:73)\n\tat org.schabi.newpipe.extractor.utils.Parser.matchGroup(Parser.java:61)\n\tat org.schabi.newpipe.extractor.utils.Parser.matchGroup1(Parser.java:52)\n\tat org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeStreamExtractor.loadDecryptionCode(YoutubeStreamExtractor.java:726)\n\t... 30 more\n"
   ],
   "user_comment": ""
}

Also adding that I was previously running 15.0. Upgrading to the latest release did not fix the issue.

[BrunoDSL]

Confirming that the error also happens in 0.15.1. Simply clicking on any video result causes the error.

[0XE4]

This was happening in 0.14.1 so I updated thinking it was that, but still occuring in 0.16 as mentioned above. Just started today.

[ghost]

probably google changed somethings again.

[Rub-z]

Indeed, there has been an update to the Youtube player; it seems to happen every few months..

It is all covered in this closed issue: https://github.com/TeamNewPipe/NewPipeExtractor/issues/133

Issue: The harcoded string DECYRYPTION_SIGNATURE_FUNCTION_REGEX needs to be updated again.

As there is currently no way to update this string, a new release will be required when it is solved

Edit. made post clearer

[regmishailendra]

Hi did you find the solution? Please tell us what should be changed?

[AnotherLife]

He said we should wait for a new release. So wait.

[regmishailendra]

hey anotherlife, can you estimate how long it may take to get new release? Just estimate

[AnotherLife]

One day.

[daufinsyd]

Since fdroid takes a few days to update, wouldn't be feasible to let the user manually enter the DECYRYPTION_SIGNATURE_FUNCTION_REGEX string ? (eg add a menu to manually set this string which could be available on the website ?)

No problem with external code and quick solution :)

[theScrabi]

Yea yea. So 0.15.2 was the first version with ota updatet. Lets se if it works now :)

[theScrabi]

However we first need to find the tight decrypt regex. Could one look over at youtube-dl and see if they have a solution already?

[theScrabi]

Issue fixed. This line here
(\\w+)\\s*=\\s*function\\((\\w+)\\)\\{\\s*\\2=\\s*\\2\\.split\\(\"\"\\)\\s*;
Should have started with this ([\\w$]+).

And this is one more reason why you should avoid regex xD

[regmishailendra]

What should we do exactly now? Pull latest version or can we change something in existing.
Please somebody suggest it.

[theScrabi]

Fix is on the way.

[theScrabi]

@TobiGr @TheAssassin we really need to get reproducible builds running, as ota is not working on fdroid yet. Could you tell me what is left to be done to get it running? Aka how should I build the app that it works? I heard I should use sortfs or similar, and must I build using fdroid?

[theScrabi]

Here is the fixed version:
https://github.com/TeamNewPipe/NewPipe/releases/tag/v0.16.1

[ekianjo]

How long until it reaches FDroid?

[theScrabi]

Cant tell, best case one day worst case one week

[Rub-z]

Thanks for the quick fix and release @theScrabi :)

[Vacuumm]

Can't there be an webbased url signature service provider so that it aint have to be hardcoded into the software? Alot of people dont know how github works and cannot manually update their software. They cannot use the app anymore when it is broken.

Thanks for the fix!

[boardfish]

I like the sound of @Vacuumm's solution. I think it should be broken down as follows:

• The most recent correct regex for the URL signature should be baked into the app as is currently the case.
• In the instance that the errors in this thread are raised, the app should call off to the service to get the new regex and update its internal config to use this if it fixes the issue.
• The service should be lightweight – as it only needs to return a single string, it might as well be a single endpoint API running off something like Sinatra/Flask.

Potentially, the endpoint URL could be a configurable option so that folks who figure out the new regex can test it without rebuilding the entire app again.

I'd raise a PR myself, but Android development isn't my corner just yet.

[Symbian9]

Can't there be an webbased url signature service provider so that it aint have to be hardcoded into the software?

This should be added as option in "Developers" settings

[TobiGr]

we really need to get reproducible builds running, as ota is not working on fdroid yet. Could you tell me what is left to be done to get it running? Aka how should I build the app that it works? I heard I should use sortfs or similar, and must I build using fdroid?

@theScrabi @TheAssassin I don't know. I'd say, we should follow the instructions in #1981 and finish our own repo. e.g. decide whether to use version names or codes to identify the correct APKs. But it is hard for me to see, what changes need to be done in the app. I'd say, our metadata file in the fdroiddata repo needs an update. This should be quite similar to the ones pointed out: Öffi and Briar

[TheAssassin]

@theScrabi @TobiGr I think we need to meet at some point (maybe even online) and discuss how we can set this up in the most easy way. We need more automation in place here, and also should try to all get the same level of knowledge on the topic.

[theScrabi]

Full ack

[stonerl]

FYI I'm still on 0.16.0 and since yesterday (8.pm UTC) I don't get the decryption error anymore.

[theScrabi]

@TheAssassin could you please set up a doodle and sent us?