Invidious: Error when attempting to log in with Google

Created on 15 Sep 2019  路  18Comments  路  Source: iv-org/invidious

Expected Array for #[](index : Int), not Int64
Traceback:
Getting lookup...done, returned 200.
Getting challenge...done, returned 400.

Official instance. Brave browser. Attempted to log in 3 times. The same error with cross-site cookies enabled.

bug

Most helpful comment

I looked at this a bit more and I suspect it has to do with the arguments to the accounts.google.com/_/signin/sl/challenge endpoint - specifically the bgRequest argument. There's hardly any information about it online. It appears to be related to "BotGuard," some kind of Google anti-spam/automation mechanism. They might have started actually verifying it's actually legitimate, which broke the previous auth method that apparently used to work by setting it to an empty string (see here). The code to generate it is found in the HTML returned by the /ServiceLogin endpoint - it's a massive wad of obfuscated JavaScript. I read somewhere that it apparently changes on every request, so it might be infeasible to calculate manually save for running it in a JavaScript VM.

I think that fixing this issue ultimately comes down to cracking Google's anti-spam protection, which I'm guessing was specifically designed not to be cracked by anyone. Alternatively there could be an input box in invidious to paste in a set of cookies from a separate login to the official YouTube frontend, if nothing else.

All 18 comments

I've pushed a couple changes with a006963fb8a03fcbc1358eb3292242cd73ba0d4a. Would you mind checking to see if you can now log in?

May be related to #755.

The error persists.

Screen Shot 2020-01-25 at 08 31 45

Duplicate of #755

this is not a duplicate, both errors are different but are linked to google it seems.
I had the issue you provided the link of.
Now it seems i have the same issue as yidakra but with Firefox

I was curious so I tried to see if youtube-dl could login with a Google account (since they seem to use the same auth process) and it turns out it can't - it also receives a 400 response from the challenge endpoint. I filed an issue here. Seems like it's been going on since around April 2019.

This thread mentions that adding cookies to the request manually works for them. I haven't tried this yet.

I looked at this a bit more and I suspect it has to do with the arguments to the accounts.google.com/_/signin/sl/challenge endpoint - specifically the bgRequest argument. There's hardly any information about it online. It appears to be related to "BotGuard," some kind of Google anti-spam/automation mechanism. They might have started actually verifying it's actually legitimate, which broke the previous auth method that apparently used to work by setting it to an empty string (see here). The code to generate it is found in the HTML returned by the /ServiceLogin endpoint - it's a massive wad of obfuscated JavaScript. I read somewhere that it apparently changes on every request, so it might be infeasible to calculate manually save for running it in a JavaScript VM.

I think that fixing this issue ultimately comes down to cracking Google's anti-spam protection, which I'm guessing was specifically designed not to be cracked by anyone. Alternatively there could be an input box in invidious to paste in a set of cookies from a separate login to the official YouTube frontend, if nothing else.

Yes, I've attempted to login to both youtube.com and accounts.google.com, downloading cookies files from both, merging them and then using youtube-dl -u [MyUserName] -p [MyPassword] --cookies cookies.txt ... and still get the 400 error.

I attempted to start a conversation on this in issue https://github.com/ytdl-org/youtube-dl/issues/23813 and couldn't find any open issues tracking this. This is the one!

I opened an issue at https://github.com/ytdl-org/youtube-dl/issues/23860 referencing all the similar issues I could find but they closed it without giving any explanation. They don't seem to be interested in addressing it.

Any news on this issue?
I would love to help out, however my knowledge here is minimal.
Facing the same thing when trying to login with my google account.

@rootedetc I think the best option is to add a new field to the Google login form in Invidious for entering a session cookie, logging in with YouTube through the official website, then providing the user with manual steps of opening the brower's developer tools and copying/pasting the cookie from the official sign-in there. Then instead of the code using the cookie that Google provides for any requests to YouTube, it uses the one pasted in.

Thanks @Ruin0x11 for that detail. That makes sense as to why https://github.com/ytdl-org/youtube-dl/issues/21313#issuecomment-499496235 is no longer working.
There is a user in this comment on Mar 24th 2020 that says it is working for him.
https://github.com/ytdl-org/youtube-dl/issues/23860#issuecomment-602834708
He, Asoretmadolell, is the only one that I can see it is allegedly working for so I suspect it was working long ago and he just has not realised it has stopped working for him.

Google is obviously making it harder to use authenticated sessions with youtube-dl and will likely only increase the frustrations.
As such I would wonder if there is a way that python could link to a browser session and profile to present prompts and captcha checks etc as required?
Of course this would necessitate I think scripts run within X on Linux or other GUI O/S.
If this could be done, it would probably make keeping up with Googles protections easier as it would be handed off to a browser profile maybe sharing the cookies files using the python webbrowser calls https://docs.python.org/3/library/webbrowser.html

I would think a dedicated browser (or at least browser profile) would be the ideal for this with probably a youtube-dl switch that chooses to utilise the external browser for google (and perhaps other future systems) auth or not.

I see they have removed other issues on this or at least blocked commenting.
I would hope that they are going to try to address this, but it if comes down to continually playing cat and mouse with googles bot brigade it sounds like it will be a challenge to keep up with unless it can somewhat be handed off to a system that google is comfortable using (ie browser).

Another option might be a greasemonkey script or similar, as a workaround. If the cookie isn't HTTP-only, which I should imagine given how much JS will be working in the background, it might be possible to display it in an alert for the user to copy into the invidious form.

i dont even press the button it just says that when I go in the websitr

Is it helpful if I say "I have this issue too"?
(coz i do, firefox 77.0.1)

"I have this issue too"

Same here, different accounts. One with normal google domain and other one with custom domains in main instance.

I noticed the same issue for me (using v2020.11.21.1), when I tried accessing my private playlists.
Using the cookie-method worked for a single page request, but not for continuations of playlists with >100 entries.

I wondered, if you could really copy a browser session that easily, while cookies like "SIDCC" and "__Secure-3PSIDCC" change with each request and probably build a request chain.

I retested with the latest update (v2020.12.2) and the login problem still exist, but I'm able to access my private playlists now fully with all continuations and over multiple youtube-dl instances, while providing a cookie-file.

For that, I logged freshly into YouTube using a different browser (for me: Chrome instead of Firefox), then halted javascript execution and exported only the youtube-cookies to a file. Then I closed Chrome.
Halting the javascript execution and not having any other YouTube-tabs loaded in that browser is probably key here!
I am still logged-in in parallel in my standard browser, while using the Chrome-session for youtube-dl.

I came across another tool, which offers to download the watchlater-playlist for you.
It solves the Google-login in an unusual way via SSO to another website, maybe you want to look into that!
That tool seem to only load the initial page and no continuations tho, so it's not useful to me as is.

I'd like to get a feedback, if the google-login is currently broken for everyone or if it depends on individual factors.
The many duplicate issues, that got created and closed in the mean time, do not contain any update on this issue either.

Was this page helpful?
0 / 5 - 0 ratings