Newpipe: no internet connection through a vpn

[subproc]

hi, using bitmask apk to connect through a vpn, no matter with which provider, i can't connect to youtube through newpipe/newpipe-beta...

[ghost]

I can connect fine to YouTube with DNS66.

[ghost]

@subproc: Do you have the same problem with other VPN apps? If so, what device, Android version and Android ROM (if not the default one) do you have?

[subproc]

using easy-vpn-free i can connect...but not with bitmask...i'm on a nexus 6p Copperhead OS (Oreo)

[ghost]

Sounds like a network config error on bitmasks side.

On 28 Feb 2018 20:00, "subproc" [e-mail] wrote:

using easy-vpn-free i can connect...but not with bitmask...i'm on a nexus
6p Copperhead OS (Oreo)

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/TeamNewPipe/NewPipe/issues/1119#issuecomment-369345880,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AH1a7l8RWzbrjoJgyVh31o8QNHTx2JcAks5tZaJkgaJpZM4SJR1L
.

[subproc]

hi, updated to newpipe beta 0.12.0 and no more connection issues...so i think we can close this issues, thanks

[robohobbit]

I get a Network Error if I search videos on NewPipe while connecting to some VPN locations using Shadowsocks (not all locations fail, and using Lantern instead of Shadowsocks works).

Meanwhile, the Trending section always works, no matter which VPN location I'm connected to.

I'm using 0.11.6 from F-Droid.

@subproc I can't find 0.12. Is it available anywhere?

[mauriciocolli]

@robohobbit marked as pre-release in the releases section.

In this beta we started using OkHttp (library instead of the built-in), so that could help.

[robohobbit]

@mauriciocolli Thanks for the swift reply. I'll have to wait then. 0.12 consistently crashes when I search for videos. Keep up the good work!

[robohobbit]

Actually, I've noticed that the problem with searching videos when on a VPN connection still exists in 0.12.

I can browse fine when I connect through the US server, but I get an error when I go through the HK server.

[mauriciocolli]

@robohobbit the latest beta crashes? Can you post the log here? (click in the share button and open on some app you can copy it) That could give some insight.

I tested NewPipe connected in ShadowSocks in the pre-defined profile and in many outer-ips, and it worked fine.

@subproc also tested BitMask and can confirm, connected to their demo server and some other provider worked fine as well.

[robohobbit]

I've been also sending the reports via email.

Here's what I've got:

{
   "user_action": "searched",
   "request": "test",
   "content_language": "GB",
   "service": "YouTube",
   "package": "org.schabi.newpipe.beta",
   "version": "0.12.0",
   "os": "Linux Android 8.1.0 - 27",
   "time": "2018-03-02 12:41",
   "ip_range": <removed>,
   "exceptions": [
      "java.lang.NullPointerException: Attempt to invoke virtual method 'org.jsoup.select.Elements org.jsoup.nodes.Element.children()' on a null object reference\n\tat org.schabi.newpipe.extractor.services.youtube.YoutubeSearchEngine.search(YoutubeSearchEngine.java:84)\n\tat org.schabi.newpipe.extractor.search.SearchResult.getSearchResult(SearchResult.java:50)\n\tat org.schabi.newpipe.util.ExtractorHelper.lambda$searchFor$0$ExtractorHelper(ExtractorHelper.java:76)\n\tat org.schabi.newpipe.util.ExtractorHelper$$Lambda$0.call(Unknown Source:10)\n\tat io.reactivex.internal.operators.single.SingleFromCallable.subscribeActual(SingleFromCallable.java:45)\n\tat io.reactivex.Single.subscribe(Single.java:2801)\n\tat io.reactivex.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.run(SingleSubscribeOn.java:89)\n\tat io.reactivex.Scheduler$DisposeTask.run(Scheduler.java:452)\n\tat io.reactivex.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:66)\n\tat io.reactivex.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java:57)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:301)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1162)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:636)\n\tat java.lang.Thread.run(Thread.java:764)\n"
   ],
   "user_comment": ""
}

[mauriciocolli]

@robohobbit It seems like that NewPipe got an unexpected page.

While connected, try to open on some browser this url (will get different user agent, but could give a clue) in both "normal" and desktop mode: https://www.youtube.com/results?q=test&page=1

[robohobbit]

@mauriciocolli I can open no problem on Chrome on my phone, both in mobile and desktop mode.

The error on NewPipe happens no matter the keyword. I've tried searching for "ted talks" and I get the same outcome if I use the HK server.

[robohobbit]

@mauriciocolli i've tried again opening the link you told me with my browser and sometimes if I connect through VPN, there's a captcha. Maybe that's what stumping NewPipe. However, SkyTube has no issue connecting.

[mauriciocolli]

I've quickly created a test app that should be able to see what is downloaded: app-release.apk.zip

It will create a file (youtube_page_test.html) in the default root (sdcard or internal storage), you can post it here if you want.

[robohobbit]

@mauriciocolli Here's the output. A captcha, as I suspected.

Response {protocol=h2, code=503, message=, url=https://ipv4.google.com/sorry/index?continue=https://www.youtube.com/results%3Fq%3Dtest%26page%3D1&q=EgQvNEfhGMHI5dQFIhkA8aeDSzg8MjDGeKBvftxorQPePN4JbtSjMgFy}
date: Fri, 02 Mar 2018 14:48:34 GMT
pragma: no-cache
expires: Fri, 01 Jan 1990 00:00:00 GMT
cache-control: no-store, no-cache, must-revalidate
content-type: text/html
server: HTTP server (unknown)
content-length: 2560
x-xss-protection: 1; mode=block
x-frame-options: SAMEORIGIN
alt-svc: hq=":443"; ma=2592000; quic=51303431; quic=51303339; quic=51303338; quic=51303337; quic=51303335,quic=":443"; ma=2592000; v="41,39,38,37,35"

About this page

Our systems have detected unusual traffic from your computer network. This page checks to see if it's really you sending the requests, and not a robot. Why did this happen?

IP address: <removed>
Time: 2018-03-02T14:48:34Z
URL: https://www.youtube.com/results?q=test&page=1

[ghost]

@mauriciocolli: It indeed is a CAPTCHA, but shouldn't the downloader catch
that with a RecaptchaException?

[mauriciocolli]

@wb9688 we only check for the code 429, you can see that it got a 503 instead.

Maybe we can fix that by checking if the response code isn't successful and the redirected url contains https://ipv4.google.com/sorry. We'll see if this approach is good enough.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[ghost]

Close this issue.

[Stypox]

See why this was closed in #3544