Seems like starting from today every video will cause this error when initialize a YouTube instance. Is there a quick way to fix this? I was using the same code yesterday and everything works fine.
python version 3.6
I've tried various pytube version such as 8.0.0, 9.5.0 and 9.5.2. None of them works and all lead to this error.
yt = YouTube('https://www.youtube.com/watch?v=p1X5A9Nmsy4')
Traceback (most recent call last):
File "", line 1, in
File "/home/bshen/anaconda3/envs/videodata/lib/python3.6/site-packages/pytube/__main__.py", line 88, in __init__
self.prefetch_init()
File "/home/bshen/anaconda3/envs/videodata/lib/python3.6/site-packages/pytube/__main__.py", line 97, in prefetch_init
self.init()
File "/home/bshen/anaconda3/envs/videodata/lib/python3.6/site-packages/pytube/__main__.py", line 130, in init
mixins.apply_descrambler(self.player_config_args, fmt)
File "/home/bshen/anaconda3/envs/videodata/lib/python3.6/site-packages/pytube/mixins.py", line 89, in apply_descrambler
for i in stream_data[key].split(',')
KeyError: 'url_encoded_fmt_stream_map'
Oh, i too got this error, in yesterday pytube normal work....
I wait a fix :)
I have the same, it was working this morning and now I get the same error.
At first, I thought It might have been my update from 9.5.1 to 9.5.2 (python 3.6) but I did that this morning and it all seemed fine at the time.
I did a quick restart and then found the issues. To me this point to a change on the youtube site?
I'm assuming that YouTube has changed something big as both ytdl-org/youtube-dl and TeamNewPipe/NewPipe are having similar issues.
All those "free youtube downloader" websites that my co-workers often use seem to be broken as well, so YouTube definitely changed something.
I found https://github.com/ytdl-org/youtube-dl/commit/bf1317d from youtube-dl, but no idea how to port the change to pytube.
Got the same error. I thought it was only on a specific set of url's but it seems to be on every one of them :((
Same here, stuck with the KeyError: 'url_encoded_fmt_stream_map' error... 😞
The strange part was that I started using pytube just a few hours ago. All of the sudden the commands that I was using were not working anymore.
Seems like this line:
if 'adaptive_fmts' in self.player_config_args:
needs to be something like
if 'adaptiveFormats' in self.player_config_args['player_response']:
But I can't quite put it together in the various places this comes up.
it seems everything works again. I have the same problem this morning, but now the same script works like before. So definitely, it's Youtube side thing.
confirm that everything is working again.
It's see stream_data without [url_encoded_fmt_stream_map] Key in mixins
Is youtube remove this key?
It seems some video has this "url_encoded_fmt_stream_map" key, and some does not.
For example:
fEnC5gwNAN0 has the key, but
BXf1j8Hz2bU does not.
So, pytube would work on the first video, but not the second video.
To find out if the video has the "url_encoded_fmt_stream_map" key, one can obtain it from a browser with the follow:
https://www.youtube.com/get_video_info?video_id={videoId}.
Sorry, that I don' t have a solution. Hopefully, get_video_info could help someone to show us a solution.
The reason for getting this KeyError is that there are many videos in YouTube now which does not have 'url_encoded_fmt_stream_map' and 'adaptive_fmts' keys in self.player_config_args.
To make this work, instead of using these keys, 'formats' and 'adaptiveFormats' keys should be used. Whole logic to make Stream object will be changed because 'url_encoded_fmt_stream_map' and 'adaptive_fmts' keys have string values while 'formats' and 'adaptiveFormats' keys are JSON objects. Hope it gets fixed.
you can replace the apply_descrambler function inside the mixins.py file with the following
if key == 'url_encoded_fmt_stream_map' and not stream_data.get('url_encoded_fmt_stream_map'):
formats = json.loads(stream_data['player_response'])['streamingData']['formats']
formats.extend(json.loads(stream_data['player_response'])['streamingData']['adaptiveFormats'])
stream_data[key] = [{u'url': format_item[u'url'],
u'type': format_item[u'mimeType'],
u'quality': format_item[u'quality'],
u'itag': format_item[u'itag']} for format_item in formats]
else:
stream_data[key] = [
{k: unquote(v) for k, v in parse_qsl(i)}
for i in stream_data[key].split(',')
]
logger.debug(
'applying descrambler\n%s',
pprint.pformat(stream_data[key], indent=2),
)
@bennyb518
Im having issues with this fix, with latest pytube im getting url not found errors on videos: DyDfgMOUjCI for example.
@bennyb518 attempted implementation of your code as above, but i'm receiving the error: KeyError: 'url' now...
Traceback (most recent call last):
File "
File "C:Program Files (x86)Python37-32libsite-packagespytube__main__.py", line 88, in __init__
self.prefetch_init()
File "C:Program Files (x86)Python37-32libsite-packagespytube__main__.py", line 97, in prefetch_init
self.init()
File "C:Program Files (x86)Python37-32libsite-packagespytube__main__.py", line 143, in init
mixins.apply_descrambler(self.player_config_args, fmt)
File "C:Program Files (x86)Python37-32libsite-packagespytubemixins.py", line 101, in apply_descrambler
u'itag': format_item[u'itag']} for format_item in formats]
File "C:Program Files (x86)Python37-32libsite-packagespytubemixins.py", line 101, in
u'itag': format_item[u'itag']} for format_item in formats]
KeyError: 'url'
I am also experiencing this issue.
Are there any solutions for this?
Thank you.
@bennyb518 tested your fix for a few videos that didn't work before and for a few that did work, and now they all worked correctly. Thanks!
@bennyb518 tested the fix too. Fixed it for a few videos, did not for some others. You should still commit your solution?
Also theres this fix, which, as someone commented, doesnt allways work, too:
https://github.com/nficano/pytube/pull/528/files
any1 tried both fixes combined?
@bennyb518 tested that fix and it seems to work
try @bennyb518 and somes video still have error because formats not include have "url" key, it replace by "cipher" so I made some change like this (I have learnt python about 2 months, so may be my code is a little long, may be someone can improve, but it work) :
import urllib.parse
if key == 'url_encoded_fmt_stream_map' and not stream_data.get('url_encoded_fmt_stream_map'):
formats = json.loads(stream_data['player_response'])['streamingData']['formats']
formats.extend(json.loads(stream_data['player_response'])['streamingData']['adaptiveFormats'])
try:
stream_data[key] = [{u'url': format_item[u'url'],
u'type': format_item[u'mimeType'],
u'quality': format_item[u'quality'],
u'itag': format_item[u'itag']} for format_item in formats]
except:
stream_data[key] = [{u'url': urllib.parse.unquote([url_item for url_item in format_item[u'cipher'].split("&") if "url=" in url_item][0].split("=")[1]),
u'sp': urllib.parse.unquote([url_item for url_item in format_item[u'cipher'].split("&") if "sp=" in url_item][0].split("=")[1]),
u's': urllib.parse.unquote([url_item for url_item in format_item[u'cipher'].split("&") if "s=" in url_item][0].split("=")[1]),
u'type': format_item[u'mimeType'],
u'quality': format_item[u'quality'],
u'itag': format_item[u'itag']} for format_item in formats]
else:
stream_data[key] = [
{k: unquote(v) for k, v in parse_qsl(i)}
for i in stream_data[key].split(',')
]
logger.debug(
'applying descrambler\n%s',
pprint.pformat(stream_data[key], indent=2),
)
Created a fix for this bug. If you want to install the package with the fix before it gets to the main package, run the following:
pip3 install git+https://github.com/giacaglia/pytube.git
@giacaglia it seems to work, thanks for forking and patching it !
I was wondering, what does the git+ before the url do ?
@giacaglia thanks, it works, but default ordering doesn't work anymore, I always have to add .order_by('resolution').desc()
I am facing an error like this
'list' object has no attribute 'split'
and the Key Error of 'url_encoded_fmt_stream_map' is not fixed
Thanks for the fix. Really appreciate it. @giacaglia
Hope it gets merged soon.
Hello !
After instaling: " pip3 install git+https://github.com/giacaglia/pytube.git"
I have this error for some links:
File "/home/jacob/.local/lib/python3.6/site-packages/pytube/__main__.py", line 88, in __init__
self.prefetch_init()
File "/home/jacob/.local/lib/python3.6/site-packages/pytube/__main__.py", line 97, in prefetch_init
self.init()
File "/home/jacob/.local/lib/python3.6/site-packages/pytube/__main__.py", line 143, in init
mixins.apply_descrambler(self.player_config_args, fmt)
File "/home/jacob/.local/lib/python3.6/site-packages/pytube/mixins.py", line 102, in apply_descrambler
u'itag': format_item[u'itag']} for format_item in formats]
File "/home/jacob/.local/lib/python3.6/site-packages/pytube/mixins.py", line 102, in
u'itag': format_item[u'itag']} for format_item in formats]
KeyError: 'url'
How can I fix this ?
still broke
after pip install git+, i'm getting KeyError: 'url' like jakub and also VideoUnavailable sometimes for v=bLfcwDZ5GzQ
@bennyb518 attempted implementation of your code as above, but i'm receiving the error: KeyError: None
Traceback (most recent call last):
File "test.py", line 16, in
for video in yt.streams.filter(only_video=True).order_by('resolution').desc().all():
File "/Users/novus/.local/share/virtualenvs/pytube-jYBLnBQZ/lib/python3.7/site-packages/pytube/query.py", line 183, in order_by
key=key,
File "/Users/novus/.local/share/virtualenvs/pytube-jYBLnBQZ/lib/python3.7/site-packages/pytube/query.py", line 176, in key
return integer_attr_repr[getattr(s, attribute_name)]
KeyError: None
still broke
after pip install git+, i'm getting KeyError: 'url' like jakub and also VideoUnavailable sometimes for v=bLfcwDZ5GzQ
True
File "/home/oscar/anaconda3/lib/python3.7/site-packages/pytube/mixins.py", line 102, in apply_descrambler
u'itag': format_item[u'itag']} for format_item in formats]
File "/home/oscar/anaconda3/lib/python3.7/site-packages/pytube/mixins.py", line 102, in
u'itag': format_item[u'itag']} for format_item in formats]
KeyError: 'url'
I had the same problem.
But I applied both of the solutions from https://github.com/nficano/pytube/issues/467#issuecomment-567560796 & https://github.com/nficano/pytube/issues/467#issuecomment-568187136. @bennyb518 @Apfelvater
They seem to work for pytube.YouTube().title.
At least I do not get errors, but clean titles for 50 "random" YouTube-Videos.
Before the error occured, I was tinkering with the YouTube-API to search for videos with certain properties.
Therefore I already had a .json-file with data from 50 different YouTube-Videos.
So I "quickly" wrote some code to test if the pytube.YouTube().title is working now - and it worked.
You find the testing script here
import json
with open('VLSresults.json', encoding='cp850') as f:
data = json.load(f)
print(data)
#key: k
#value dict[k]
yt_base = 'https://www.youtube.com/watch?v='
link_list = []
for item in data['items']:
#print(item['id'])
yt_id = item['id']
yt_link = yt_base + yt_id
link_list.append(yt_link)
print(link_list)
from pytube import YouTube
for link in link_list:
yt = YouTube(link)
print(yt.title)
Here you find an example for an item from the .json-data which I got as a response to my search with the YouTube-data-api
{
"kind": "youtube#videoListResponse",
"etag": "\"p4VTdlkQv3HQeTEaXgvLePAydmU/RiCiued1hbJOsx-2sWdAgKSPS1Y\"",
"nextPageToken": "CDIQAA",
"pageInfo": {
"totalResults": 200,
"resultsPerPage": 50
},
"items": [
{
"kind": "youtube#video",
"etag": "\"p4VTdlkQv3HQeTEaXgvLePAydmU/cFOeTN3T-zesgTDAn82gpOT_Xbw\"",
"id": "Rw1khSnj24s",
"snippet": {
"publishedAt": "2019-12-27T20:22:00.000Z",
"channelId": "UCoeOLs0_MxLMhHzTJNK4Xfw",
"title": "500.000 Likes...",
"description": "what a time to be alive",
"thumbnails": {
"default": {
"url": "https://i.ytimg.com/vi/Rw1khSnj24s/default.jpg",
"width": 120,
"height": 90
},
"medium": {
"url": "https://i.ytimg.com/vi/Rw1khSnj24s/mqdefault.jpg",
"width": 320,
"height": 180
},
"high": {
"url": "https://i.ytimg.com/vi/Rw1khSnj24s/hqdefault.jpg",
"width": 480,
"height": 360
},
"standard": {
"url": "https://i.ytimg.com/vi/Rw1khSnj24s/sddefault.jpg",
"width": 640,
"height": 480
},
"maxres": {
"url": "https://i.ytimg.com/vi/Rw1khSnj24s/maxresdefault.jpg",
"width": 1280,
"height": 720
}
},
"channelTitle": "OHMEYGOD",
"tags": [
"unge",
"simon",
"tattoo",
"bulien jam",
"julien bam",
"knossi"
],
"categoryId": "24",
"liveBroadcastContent": "none",
"localized": {
"title": "500.000 Likes...",
"description": "what a time to be alive"
}
},
"contentDetails": {
"duration": "PT3M2S",
"dimension": "2d",
"definition": "hd",
"caption": "false",
"licensedContent": true,
"projection": "rectangular"
},
"status": {
"uploadStatus": "processed",
"privacyStatus": "public",
"license": "youtube",
"embeddable": true,
"publicStatsViewable": true
},
"statistics": {
"viewCount": "630225",
"likeCount": "69751",
"dislikeCount": "1759",
"favoriteCount": "0",
"commentCount": "3223"
}
},
{
<i> HERE YOU WOULD FIND THE NEXT ITEM... </i>
}
}
]
}
I had the same problem.
But I applied both of the solutions from #467 (comment) & #467 (comment). @bennyb518 @Apfelvater
They seem to work for pytube.YouTube().title.
At least I do not get errors, but clean titles for 50 "random" YouTube-Videos.What I did:
Before the error occured, I was tinkering with the YouTube-API to search for videos with certain properties.
Therefore I already had a .json-file with data from 50 different YouTube-Videos.
So I "quickly" wrote some code to test if the pytube.YouTube().title is working now - and it worked.
You find the testing script here
Here you find an example for an item from the .json-data which I got as a response to my search with the YouTube-data-api
thanks
that works for me¡
When will the fix get pulled into official binaries?
Sure, there is no error :D
But only a title show's up and I have no streams to download
I implemented the code provided by @FidoDidoVN , and it works very well.
All we need to do is to replace the code provided by @FidoDidoVN in the apply_descrambler function within the mixins.py file.
try @bennyb518 and somes video still have error because formats not include have "url" key, it replace by "cipher" so I made some change like this (I have learnt python about 2 months, so may be my code is a little long, may be someone can improve, but it work) :
import urllib.parse if key == 'url_encoded_fmt_stream_map' and not stream_data.get('url_encoded_fmt_stream_map'): formats = json.loads(stream_data['player_response'])['streamingData']['formats'] formats.extend(json.loads(stream_data['player_response'])['streamingData']['adaptiveFormats']) try: stream_data[key] = [{u'url': format_item[u'url'], u'type': format_item[u'mimeType'], u'quality': format_item[u'quality'], u'itag': format_item[u'itag']} for format_item in formats] except: stream_data[key] = [{u'url': urllib.parse.unquote([url_item for url_item in format_item[u'cipher'].split("&") if "url=" in url_item][0].split("=")[1]), u'sp': urllib.parse.unquote([url_item for url_item in format_item[u'cipher'].split("&") if "sp=" in url_item][0].split("=")[1]), u's': urllib.parse.unquote([url_item for url_item in format_item[u'cipher'].split("&") if "s=" in url_item][0].split("=")[1]), u'type': format_item[u'mimeType'], u'quality': format_item[u'quality'], u'itag': format_item[u'itag']} for format_item in formats] else: stream_data[key] = [ {k: unquote(v) for k, v in parse_qsl(i)} for i in stream_data[key].split(',') ] logger.debug( 'applying descrambler\n%s', pprint.pformat(stream_data[key], indent=2), )
hello!
After fixing code as above, I meet a new issue:
AttributeError: 'YouTube' object has no attribute 'download'
How can I fix this? Thank you!
try @bennyb518 and somes video still have error because formats not include have "url" key, it replace by "cipher" so I made some change like this (I have learnt python about 2 months, so may be my code is a little long, may be someone can improve, but it work) :
import urllib.parse if key == 'url_encoded_fmt_stream_map' and not stream_data.get('url_encoded_fmt_stream_map'): formats = json.loads(stream_data['player_response'])['streamingData']['formats'] formats.extend(json.loads(stream_data['player_response'])['streamingData']['adaptiveFormats']) try: stream_data[key] = [{u'url': format_item[u'url'], u'type': format_item[u'mimeType'], u'quality': format_item[u'quality'], u'itag': format_item[u'itag']} for format_item in formats] except: stream_data[key] = [{u'url': urllib.parse.unquote([url_item for url_item in format_item[u'cipher'].split("&") if "url=" in url_item][0].split("=")[1]), u'sp': urllib.parse.unquote([url_item for url_item in format_item[u'cipher'].split("&") if "sp=" in url_item][0].split("=")[1]), u's': urllib.parse.unquote([url_item for url_item in format_item[u'cipher'].split("&") if "s=" in url_item][0].split("=")[1]), u'type': format_item[u'mimeType'], u'quality': format_item[u'quality'], u'itag': format_item[u'itag']} for format_item in formats] else: stream_data[key] = [ {k: unquote(v) for k, v in parse_qsl(i)} for i in stream_data[key].split(',') ] logger.debug( 'applying descrambler\n%s', pprint.pformat(stream_data[key], indent=2), )
thank you,it works
@ValentiaSulli can you show your code? may be your code is wrong. 'YouTube' object has no attribute 'download', you have to use this code: YouTube(link video).streams.first().download(yours path). For more advanced you can google how to download youtube video by pytube
I get this Error after benny518's solution:
Traceback (most recent call last):
File "ytdl.py", line 27, in <module>
main()
File "ytdl.py", line 16, in main
download(url)
File "ytdl.py", line 20, in download
YouTube(url).streams.first().download(dlpath)
File "/anaconda3/lib/python3.6/site-packages/pytube/__main__.py", line 88, in __init__
self.prefetch_init()
File "/anaconda3/lib/python3.6/site-packages/pytube/__main__.py", line 97, in prefetch_init
self.init()
File "/anaconda3/lib/python3.6/site-packages/pytube/__main__.py", line 143, in init
mixins.apply_descrambler(self.player_config_args, fmt)
File "/anaconda3/lib/python3.6/site-packages/pytube/mixins.py", line 102, in apply_descrambler
u'itag': format_item[u'itag']} for format_item in formats]
File "/anaconda3/lib/python3.6/site-packages/pytube/mixins.py", line 102, in <listcomp>
u'itag': format_item[u'itag']} for format_item in formats]
KeyError: 'url'
try @ bennyb518 e o somes video ainda tem erro porque os formatos não incluem a chave "url", substituem por "cipher", por isso fiz algumas alterações como essa (aprendi python há cerca de 2 meses, portanto, meu código é um pouco longo , pode ser que alguém possa melhorar, mas funciona):
import urllib.parse if key == 'url_encoded_fmt_stream_map' and not stream_data.get('url_encoded_fmt_stream_map'): formats = json.loads(stream_data['player_response'])['streamingData']['formats'] formats.extend(json.loads(stream_data['player_response'])['streamingData']['adaptiveFormats']) try: stream_data[key] = [{u'url': format_item[u'url'], u'type': format_item[u'mimeType'], u'quality': format_item[u'quality'], u'itag': format_item[u'itag']} for format_item in formats] except: stream_data[key] = [{u'url': urllib.parse.unquote([url_item for url_item in format_item[u'cipher'].split("&") if "url=" in url_item][0].split("=")[1]), u'sp': urllib.parse.unquote([url_item for url_item in format_item[u'cipher'].split("&") if "sp=" in url_item][0].split("=")[1]), u's': urllib.parse.unquote([url_item for url_item in format_item[u'cipher'].split("&") if "s=" in url_item][0].split("=")[1]), u'type': format_item[u'mimeType'], u'quality': format_item[u'quality'], u'itag': format_item[u'itag']} for format_item in formats] else: stream_data[key] = [ {k: unquote(v) for k, v in parse_qsl(i)} for i in stream_data[key].split(',') ] logger.debug( 'applying descrambler\n%s', pprint.pformat(stream_data[key], indent=2), )
Funcionou perfeitamente!
Worked perfectly!
TKS from Brazil
Thank you @FidoDidoVN. This Solution Works for me :)
greetings from switzerland
Just wondering if someone could provide a PR to integrate the fix upstream? Manually patching source code is a quick workaround only.
Just wondering if someone could provide a PR to integrate the fix upstream? Manually patching source code is a quick workaround only.
I wouldn't count on it any time soon. This repo almost seems abandoned. The last commit was in October. There are 151 issues and 49 PRs. This issue has been open since September. But the owner was making commits on other repos 3 days ago.
Just wondering if someone could provide a PR to integrate the fix upstream? Manually patching source code is a quick workaround only.
I wouldn't count on it any time soon. This repo almost seems abandoned. The last commit was in October. There are 151 issues and 49 PRs. This issue has been open since September. But the owner was making commits on other repos 3 days ago.
I was wondering about this as well.
@nficano - are you the only maintainer? Is there a current status of the project? If it's 'abandoned' or 'looking for maintainers' - maybe this could be reflected in the README file, and maybe others are interested in taking over/helping out...
fixed in my Python 3 only fork: https://github.com/hbmartin/pytube3
fixed in my Python 3 only fork: https://github.com/hbmartin/pytube3
File "C:\Users\...\AppData\Local\Programs\Python\Python37\lib\site-packages\pytube\cipher.py", line 271, in get_signature
var, _ = transform_plan[0].split(".")
ValueError: too many values to unpack (expected 2)
@hbmartin i am getting this error using your fork
@pintonos this is a recent YouTube change, tracking work in: https://github.com/hbmartin/pytube3/issues/12
I am getting this KeyError: 'url' return after implementing @bennyb518's code. Anyone know what causes this issue and most importantly how to fix it?
clone repo from https://github.com/hbmartin/pytube3
and paste pytube folder from the repo to the site-packages directory, everything should work fine
@braindotai Thanks!
paste pytube folder from the repo to the site-packages directory
Please don't do this. It will prevent you from getting pytube3 upgrades and will probably lead to weird bugs. Better to ‘pip uninstall pytube; pip install pytube3’
paste pytube folder from the repo to the site-packages directory
Please don't do this. It will prevent you from getting pytube3 upgrades and will probably lead to weird bugs. Better to ‘pip uninstall pytube; pip install pytube3’
That's correct 👍
@bennyb518 it still returns error please help me out
stream_data[key] = [{u'url': format_item[u'url'],
KeyError: 'url'
I combined the two solutions in this thread as well as added support for another error:
mixins.apply_descrambler(self.player_config_args, fmt)
File "/home/kerri/Documents/Projects/crowdCounting/crowd/lib/python3.6/site-packages/pytube/mixins.py", line 97, in apply_descrambler
formats = json.loads(stream_data['player_response'])['streamingData']['formats']
KeyError: 'formats'
Pull Requests have been made to both this repo and that of @giacaglia. As of writing the git+https://github.com/giacaglia/pytube.git is pulled to master.
@krapes PR is now merged to my repo. Thanks @krapes for the PR
ImportError: cannot import name 'quote' from 'pytube.compat' (/home/islam/projects/youtube/venv/lib/python3.7/site-packages/pytube/compat.py)
I combined the two solutions in this thread as well as added support for another error:
mixins.apply_descrambler(self.player_config_args, fmt)
File "/home/kerri/Documents/Projects/crowdCounting/crowd/lib/python3.6/site-packages/pytube/mixins.py", line 97, in apply_descrambler
formats = json.loads(stream_data['player_response'])['streamingData']['formats']
KeyError: 'formats'Pull Requests have been made to both this repo and that of @giacaglia. As of writing the
git+https://github.com/giacaglia/pytube.gitis pulled to master.
I hereby thank you officially in the name of the whole planet.
pip3 install git+https://github.com/giacaglia/pytube.git --upgrade
Most helpful comment
you can replace the apply_descrambler function inside the mixins.py file with the following