Newpipe: Invalid json from bug report

Created on 28 Dec 2019 · 5Comments · Source: TeamNewPipe/NewPipe

[x] I carefully read the contribution guidelines and agree to them.
[x] I checked if the issue/feature exists in the latest version.
[x] I did use the incredible bugreport to markdown converter to paste bug reports.

As @eladkarako stated in https://github.com/TeamNewPipe/NewPipe/issues/2867#issuecomment-569373332, the bug-report-to-markdown-converter couldn't convert a bug report / crash log generated by NewPipe. This is due to some unescaped " inside the exception string: [...] host "youtube.com": No [...]. After escaping correctly ([...] host \"youtube.com\": No [...]) the bug report could be converted correctly.

Original bug report generated by NewPipe

{
"user_action": "requested stream",
"request": "https://www.youtube.com/watch?v=7im9BFEHzEs",
"content_language": "GB",
"service": "YouTube",
"package": "org.schabi.newpipe",
"version": "0.17.4",
"os": "Linux Android 9 - 28",
"time": "2019-12-14 19:44",
"exceptions": [
"org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeStreamExtractor$DecryptException: Could not load decrypt function\n\tat org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeStreamExtractor.loadDecryptionCode(YoutubeStreamExtractor.java:777)\n\tat org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeStreamExtractor.onFetchPage(YoutubeStreamExtractor.java:646)\n\tat org.schabi.newpipe.extractor.Extractor.fetchPage(Extractor.java:52)\n\tat org.schabi.newpipe.extractor.stream.StreamInfo.getInfo(StreamInfo.java:63)\n\tat org.schabi.newpipe.extractor.stream.StreamInfo.getInfo(StreamInfo.java:59)\n\tat org.schabi.newpipe.util.ExtractorHelper.lambda$getStreamInfo$3(ExtractorHelper.java:115)\n\tat org.schabi.newpipe.util.-$$Lambda$ExtractorHelper$5fJcha6Sq5APJBLdG6osaJby-mc.call(Unknown Source:4)\n\tat io.reactivex.internal.operators.single.SingleFromCallable.subscribeActual(SingleFromCallable.java:44)\n\tat io.reactivex.Single.subscribe(Single.java:3438)\n\tat io.reactivex.internal.operators.single.SingleDoOnSuccess.subscribeActual(SingleDoOnSuccess.java:35)\n\tat io.reactivex.Single.subscribe(Single.java:3438)\n\tat io.reactivex.internal.operators.maybe.MaybeFromSingle.subscribeActual(MaybeFromSingle.java:41)\n\tat io.reactivex.Maybe.subscribe(Maybe.java:4154)\n\tat io.reactivex.internal.operators.maybe.MaybeConcatArray$ConcatMaybeObserver.drain(MaybeConcatArray.java:153)\n\tat io.reactivex.internal.operators.maybe.MaybeConcatArray$ConcatMaybeObserver.request(MaybeConcatArray.java:78)\n\tat io.reactivex.internal.operators.flowable.FlowableElementAtMaybe$ElementAtSubscriber.onSubscribe(FlowableElementAtMaybe.java:66)\n\tat io.reactivex.internal.operators.maybe.MaybeConcatArray.subscribeActual(MaybeConcatArray.java:42)\n\tat io.reactivex.Flowable.subscribe(Flowable.java:14479)\n\tat io.reactivex.internal.operators.flowable.FlowableElementAtMaybe.subscribeActual(FlowableElementAtMaybe.java:36)\n\tat io.reactivex.Maybe.subscribe(Maybe.java:4154)\n\tat io.reactivex.internal.operators.maybe.MaybeToSingle.subscribeActual(MaybeToSingle.java:46)\n\tat io.reactivex.Single.subscribe(Single.java:3438)\n\tat io.reactivex.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.run(SingleSubscribeOn.java:89)\n\tat io.reactivex.Scheduler$DisposeTask.run(Scheduler.java:578)\n\tat io.reactivex.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:66)\n\tat io.reactivex.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java:57)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:301)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)\n\tat java.lang.Thread.run(Thread.java:764)\nCaused by: java.net.UnknownHostException: Unable to resolve host "youtube.com": No address associated with hostname\n\tat java.net.Inet6AddressImpl.lookupHostByName(Inet6AddressImpl.java:157)\n\tat java.net.Inet6AddressImpl.lookupAllHostAddr(Inet6AddressImpl.java:105)\n\tat java.net.InetAddress.getAllByName(InetAddress.java:1154)\n\tat okhttp3.Dns$1.lookup(Dns.java:40)\n\tat okhttp3.internal.connection.RouteSelector.resetNextInetSocketAddress(RouteSelector.java:185)\n\tat okhttp3.internal.connection.RouteSelector.nextProxy(RouteSelector.java:149)\n\tat okhttp3.internal.connection.RouteSelector.next(RouteSelector.java:84)\n\tat okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:215)\n\tat okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:135)\n\tat okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:114)\n\tat okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)\n\tat okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)\n\tat okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)\n\tat okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)\n\tat okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)\n\tat okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)\n\tat okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)\n\tat okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)\n\tat okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:127)\n\tat okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)\n\tat okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)\n\tat okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:257)\n\tat okhttp3.RealCall.execute(RealCall.java:93)\n\tat org.schabi.newpipe.Downloader.getBody(Downloader.java:162)\n\tat org.schabi.newpipe.Downloader.download(Downloader.java:134)\n\tat org.schabi.newpipe.Downloader.download(Downloader.java:186)\n\tat org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeStreamExtractor.loadDecryptionCode(YoutubeStreamExtractor.java:757)\n\t... 30 more\nCaused by: android.system.GaiException: android_getaddrinfo failed: EAI_NODATA (No address associated with hostname)\n\tat libcore.io.Linux.android_getaddrinfo(Native Method)\n\tat libcore.io.BlockGuardOs.android_getaddrinfo(BlockGuardOs.java:172)\n\tat java.net.Inet6AddressImpl.lookupHostByName(Inet6AddressImpl.java:137)\n\t... 56 more\n"
],
"user_comment": ""
}

Fixed bug report

{
"user_action": "requested stream",
"request": "https://www.youtube.com/watch?v=7im9BFEHzEs",
"content_language": "GB",
"service": "YouTube",
"package": "org.schabi.newpipe",
"version": "0.17.4",
"os": "Linux Android 9 - 28",
"time": "2019-12-14 19:44",
"exceptions": [
"org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeStreamExtractor$DecryptException: Could not load decrypt function\n\tat org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeStreamExtractor.loadDecryptionCode(YoutubeStreamExtractor.java:777)\n\tat org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeStreamExtractor.onFetchPage(YoutubeStreamExtractor.java:646)\n\tat org.schabi.newpipe.extractor.Extractor.fetchPage(Extractor.java:52)\n\tat org.schabi.newpipe.extractor.stream.StreamInfo.getInfo(StreamInfo.java:63)\n\tat org.schabi.newpipe.extractor.stream.StreamInfo.getInfo(StreamInfo.java:59)\n\tat org.schabi.newpipe.util.ExtractorHelper.lambda$getStreamInfo$3(ExtractorHelper.java:115)\n\tat org.schabi.newpipe.util.-$$Lambda$ExtractorHelper$5fJcha6Sq5APJBLdG6osaJby-mc.call(Unknown Source:4)\n\tat io.reactivex.internal.operators.single.SingleFromCallable.subscribeActual(SingleFromCallable.java:44)\n\tat io.reactivex.Single.subscribe(Single.java:3438)\n\tat io.reactivex.internal.operators.single.SingleDoOnSuccess.subscribeActual(SingleDoOnSuccess.java:35)\n\tat io.reactivex.Single.subscribe(Single.java:3438)\n\tat io.reactivex.internal.operators.maybe.MaybeFromSingle.subscribeActual(MaybeFromSingle.java:41)\n\tat io.reactivex.Maybe.subscribe(Maybe.java:4154)\n\tat io.reactivex.internal.operators.maybe.MaybeConcatArray$ConcatMaybeObserver.drain(MaybeConcatArray.java:153)\n\tat io.reactivex.internal.operators.maybe.MaybeConcatArray$ConcatMaybeObserver.request(MaybeConcatArray.java:78)\n\tat io.reactivex.internal.operators.flowable.FlowableElementAtMaybe$ElementAtSubscriber.onSubscribe(FlowableElementAtMaybe.java:66)\n\tat io.reactivex.internal.operators.maybe.MaybeConcatArray.subscribeActual(MaybeConcatArray.java:42)\n\tat io.reactivex.Flowable.subscribe(Flowable.java:14479)\n\tat io.reactivex.internal.operators.flowable.FlowableElementAtMaybe.subscribeActual(FlowableElementAtMaybe.java:36)\n\tat io.reactivex.Maybe.subscribe(Maybe.java:4154)\n\tat io.reactivex.internal.operators.maybe.MaybeToSingle.subscribeActual(MaybeToSingle.java:46)\n\tat io.reactivex.Single.subscribe(Single.java:3438)\n\tat io.reactivex.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.run(SingleSubscribeOn.java:89)\n\tat io.reactivex.Scheduler$DisposeTask.run(Scheduler.java:578)\n\tat io.reactivex.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:66)\n\tat io.reactivex.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java:57)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:301)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)\n\tat java.lang.Thread.run(Thread.java:764)\nCaused by: java.net.UnknownHostException: Unable to resolve host \"youtube.com\": No address associated with hostname\n\tat java.net.Inet6AddressImpl.lookupHostByName(Inet6AddressImpl.java:157)\n\tat java.net.Inet6AddressImpl.lookupAllHostAddr(Inet6AddressImpl.java:105)\n\tat java.net.InetAddress.getAllByName(InetAddress.java:1154)\n\tat okhttp3.Dns$1.lookup(Dns.java:40)\n\tat okhttp3.internal.connection.RouteSelector.resetNextInetSocketAddress(RouteSelector.java:185)\n\tat okhttp3.internal.connection.RouteSelector.nextProxy(RouteSelector.java:149)\n\tat okhttp3.internal.connection.RouteSelector.next(RouteSelector.java:84)\n\tat okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:215)\n\tat okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:135)\n\tat okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:114)\n\tat okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)\n\tat okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)\n\tat okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)\n\tat okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)\n\tat okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)\n\tat okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)\n\tat okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)\n\tat okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)\n\tat okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:127)\n\tat okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)\n\tat okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)\n\tat okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:257)\n\tat okhttp3.RealCall.execute(RealCall.java:93)\n\tat org.schabi.newpipe.Downloader.getBody(Downloader.java:162)\n\tat org.schabi.newpipe.Downloader.download(Downloader.java:134)\n\tat org.schabi.newpipe.Downloader.download(Downloader.java:186)\n\tat org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeStreamExtractor.loadDecryptionCode(YoutubeStreamExtractor.java:757)\n\t... 30 more\nCaused by: android.system.GaiException: android_getaddrinfo failed: EAI_NODATA (No address associated with hostname)\n\tat libcore.io.Linux.android_getaddrinfo(Native Method)\n\tat libcore.io.BlockGuardOs.android_getaddrinfo(BlockGuardOs.java:172)\n\tat java.net.Inet6AddressImpl.lookupHostByName(Inet6AddressImpl.java:137)\n\t... 56 more\n"
],
"user_comment": ""
}

Fixed bug report converted to Markdown using this

Exception

__User Action:__ requested stream
__Request:__ https://www.youtube.com/watch?v=7im9BFEHzEs
__Content Language:__ GB
__Service:__ YouTube
__Version:__ 0.17.4
__OS:__ Linux Android 9 - 28

Crash log

org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeStreamExtractor$DecryptException: Could not load decrypt function
    at org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeStreamExtractor.loadDecryptionCode(YoutubeStreamExtractor.java:777)
    at org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeStreamExtractor.onFetchPage(YoutubeStreamExtractor.java:646)
    at org.schabi.newpipe.extractor.Extractor.fetchPage(Extractor.java:52)
    at org.schabi.newpipe.extractor.stream.StreamInfo.getInfo(StreamInfo.java:63)
    at org.schabi.newpipe.extractor.stream.StreamInfo.getInfo(StreamInfo.java:59)
    at org.schabi.newpipe.util.ExtractorHelper.lambda$getStreamInfo$3(ExtractorHelper.java:115)
    at org.schabi.newpipe.util.-$$Lambda$ExtractorHelper$5fJcha6Sq5APJBLdG6osaJby-mc.call(Unknown Source:4)
    at io.reactivex.internal.operators.single.SingleFromCallable.subscribeActual(SingleFromCallable.java:44)
    at io.reactivex.Single.subscribe(Single.java:3438)
    at io.reactivex.internal.operators.single.SingleDoOnSuccess.subscribeActual(SingleDoOnSuccess.java:35)
    at io.reactivex.Single.subscribe(Single.java:3438)
    at io.reactivex.internal.operators.maybe.MaybeFromSingle.subscribeActual(MaybeFromSingle.java:41)
    at io.reactivex.Maybe.subscribe(Maybe.java:4154)
    at io.reactivex.internal.operators.maybe.MaybeConcatArray$ConcatMaybeObserver.drain(MaybeConcatArray.java:153)
    at io.reactivex.internal.operators.maybe.MaybeConcatArray$ConcatMaybeObserver.request(MaybeConcatArray.java:78)
    at io.reactivex.internal.operators.flowable.FlowableElementAtMaybe$ElementAtSubscriber.onSubscribe(FlowableElementAtMaybe.java:66)
    at io.reactivex.internal.operators.maybe.MaybeConcatArray.subscribeActual(MaybeConcatArray.java:42)
    at io.reactivex.Flowable.subscribe(Flowable.java:14479)
    at io.reactivex.internal.operators.flowable.FlowableElementAtMaybe.subscribeActual(FlowableElementAtMaybe.java:36)
    at io.reactivex.Maybe.subscribe(Maybe.java:4154)
    at io.reactivex.internal.operators.maybe.MaybeToSingle.subscribeActual(MaybeToSingle.java:46)
    at io.reactivex.Single.subscribe(Single.java:3438)
    at io.reactivex.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.run(SingleSubscribeOn.java:89)
    at io.reactivex.Scheduler$DisposeTask.run(Scheduler.java:578)
    at io.reactivex.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:66)
    at io.reactivex.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java:57)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:301)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
    at java.lang.Thread.run(Thread.java:764)
Caused by: java.net.UnknownHostException: Unable to resolve host "youtube.com": No address associated with hostname
    at java.net.Inet6AddressImpl.lookupHostByName(Inet6AddressImpl.java:157)
    at java.net.Inet6AddressImpl.lookupAllHostAddr(Inet6AddressImpl.java:105)
    at java.net.InetAddress.getAllByName(InetAddress.java:1154)
    at okhttp3.Dns$1.lookup(Dns.java:40)
    at okhttp3.internal.connection.RouteSelector.resetNextInetSocketAddress(RouteSelector.java:185)
    at okhttp3.internal.connection.RouteSelector.nextProxy(RouteSelector.java:149)
    at okhttp3.internal.connection.RouteSelector.next(RouteSelector.java:84)
    at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:215)
    at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:135)
    at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:114)
    at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
    at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
    at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
    at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:127)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
    at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:257)
    at okhttp3.RealCall.execute(RealCall.java:93)
    at org.schabi.newpipe.Downloader.getBody(Downloader.java:162)
    at org.schabi.newpipe.Downloader.download(Downloader.java:134)
    at org.schabi.newpipe.Downloader.download(Downloader.java:186)
    at org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeStreamExtractor.loadDecryptionCode(YoutubeStreamExtractor.java:757)
    ... 30 more
Caused by: android.system.GaiException: android_getaddrinfo failed: EAI_NODATA (No address associated with hostname)
    at libcore.io.Linux.android_getaddrinfo(Native Method)
    at libcore.io.BlockGuardOs.android_getaddrinfo(BlockGuardOs.java:172)
    at java.net.Inet6AddressImpl.lookupHostByName(Inet6AddressImpl.java:137)
    ... 56 more

bug good first issue

Source

Stypox

All 5 comments

nice catch on those unescaped characters.

eladkarako on 28 Dec 2019

b.t.w.
Escaping those characters should be probably done in the NewPipe exception-handler method,
and not in https://github.com/TeamNewPipe/CrashReportToMarkdown
_{(although I guess the last would be somewhat easier... with just pre-replacing " to " or "\\u0022"...)}

eladkarako on 28 Dec 2019

Yeah, that's why I reported the issue here ;-)

Stypox on 29 Dec 2019

👍1

This has to be an issue with the library we are using (org.json), because what the code does in ErrorActivity.java is just converting every Throwable's stacktrace to an unescaped String, and then adding this string to a JSONArray (which automatically escapes everything). So if " don't get escaped but other things do, I guess this is a problem with the library

Stypox on 20 Apr 2020

👍1

I've created a branch to use nanojson few days ago, I'll make a PR in a short time