Spotify-downloader: KeyError: 'cipher': Most tracks fail to download
What is the purpose of your issue?
- [x] Bug
- [ ] Feature Request
- [ ] Question
- [ ] Other
Description
When I try to download a playlist spotdl will download one song and then crash.
I generated a playlist .txt file using the spodl --playlist command, but when I run the spotdl --list command only one song will download. Then the entire program will crash after attempting to fetch the metadata of the next song. Running the command again immediatly crashes and shows the same errors.
If I wait for a few minutes before running the command, it will again download one song and crash after attempting to fetch the metadata of the next song.
Log
$ spotdl --list well-this-is-me.txt -ll debug
DEBUG:spotdl.command_line.core:45:
Received arguments:
{'song': None, 'list': 'well-this-is-me.txt', 'playlist': None, 'album': None, 'all_albums': None, 'username': None, 'write_m3u': False, 'manual': False, 'no_metadata': False, 'no_encode': False, 'overwrite': 'prompt', 'quality': 'best', 'input_ext': 'automatic', 'output_ext': 'mp3', 'write_to': None, 'output_file': '{artist} - {track-name}.{output-ext}', 'trim_silence': False, 'search_format': '{artist} - {track-name} lyrics', 'dry_run': False, 'processor': 'synchronous', 'no_spaces': False, 'skip_file': None, 'write_successful_file': None, 'spotify_client_id': '4fe3fecfe5334023a1472516cc99d805', 'spotify_client_secret': '0f02b7c483c04257984695007a4a8d5c', 'log_level': 'DEBUG', 'config': '/home/eeffeemorgan/.config/spotdl/config.yml', 'remove_config': False}
DEBUG:spotdl.authorize.services.spotify:37:
Setting master Spotify credentials.
DEBUG:spotdl.authorize.services.spotify:32:
Reading cached master Spotify credentials.
INFO:spotdl.command_line.core:315:
Checking and removing any duplicate tracks in "well-this-is-me.txt".
INFO:spotdl.command_line.core:330:
Downloading 387 tracks.
DEBUG:spotdl.authorize.services.spotify:32:
Reading cached master Spotify credentials.
INFO:spotdl.command_line.core:344:
1. Downloading "https://open.spotify.com/track/0SuLAslEMFZAXf0SwY7syi"
DEBUG:spotdl.metadata_search:182:
Extracting YouTube and Spotify metadata for input Spotify URI.
DEBUG:spotdl.metadata_search:248:
Extracting Spotify metadata for input Spotify URI.
DEBUG:spotdl.metadata.providers.spotify:25:
Fetching Spotify metadata for "https://open.spotify.com/track/0SuLAslEMFZAXf0SwY7syi".
DEBUG:spotdl.metadata_search:248:
Extracting Spotify metadata for input Spotify URI.
DEBUG:spotdl.metadata.providers.spotify:25:
Fetching Spotify metadata for "https://open.spotify.com/track/0SuLAslEMFZAXf0SwY7syi".
DEBUG:spotdl.metadata.providers.youtube:119:
Fetching YouTube results for "Jon Bellion - All Time Low lyrics" at "https://www.youtube.com/results?sp=EgIQAQ%253D%253D&q=Jon%20Bellion%20-%20All%20Time%20Low%20lyrics".
DEBUG:spotdl.metadata.providers.youtube:43:
Matched with: JON BELLION - All time low (Lyrics) (https://www.youtube.com/watch?v=YAScrYlQuNo) [3:53]
DEBUG:spotdl.metadata.providers.youtube:225:
Fetching YouTube metadata for "https://www.youtube.com/watch?v=YAScrYlQuNo".
DEBUG:spotdl.metadata.providers.youtube:236:
YouTube returned an unexpected response for `pytube.YouTube(https://www.youtube.com/watch?v=YAScrYlQuNo)`. Retries left: 4
DEBUG:spotdl.metadata.providers.youtube:225:
Fetching YouTube metadata for "https://www.youtube.com/watch?v=YAScrYlQuNo".
DEBUG:spotdl.metadata.providers.youtube:194:
Selected best quality stream for "automatic" format:
{'bitrate': 160, 'connection': <http.client.HTTPResponse object at 0x7f464089f3d0>, 'download_url': 'https://r6---sn-qxoedn7z.googlevideo.com/videoplayback?expire=1590739933&ei=fW_QXrOuDoKYkwb8h4T4Dg&ip=65.130.15.208&id=o-AN18QXLOIFcVnsjaPLrUZxThmEU_9XrCLkC5ffI8EUWG&itag=251&source=youtube&requiressl=yes&mh=rx&mm=31%2C29&mn=sn-qxoedn7z%2Csn-qxo7rn7e&ms=au%2Crdu&mv=m&mvi=5&pl=16&initcwndbps=830000&vprv=1&mime=audio%2Fwebm&gir=yes&clen=3793842&dur=232.721&lmt=1590173125875195&mt=1590718252&fvip=6&keepalive=yes&c=WEB&txp=5431432&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cvprv%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRAIgXkf7Frdgh7GM39f3bbkmxLVhVKYdnrn9BrsFjJPCpxUCIFGGOOmDJDRAmMlS7Bag1RpEXPNRtQcKSTmUaDlPW-Tn&sig=AOq0QJ8wRgIhAJ6gULtSrfdlb6zJacyxuAhl1Vx7itc64uXxTrttEFJiAiEAxn3-jHHWHiI9S1wUM40oo1DSr_3Nnkyq9_x2FGheuII=', 'encoding': 'opus', 'filesize': 3793842}
DEBUG:spotdl.lyrics.providers.genius:107:
Fetching lyrics for the search query on "Jon Bellion - All Time Low".
DEBUG:spotdl.lyrics.providers.genius:82:
Fetching Genius search results from "https://genius.com/api/search/multi?per_page=1&q=Jon%2BBellion%2B-%2BAll%2BTime%2BLow".
INFO:spotdl.command_line.core:266:
Downloading to "Jon Bellion - All Time Low.mp3"
DEBUG:spotdl.encode.encoders.ffmpeg:108:
Calling FFmpeg with:
['ffmpeg', '-y', '-nostdin', '-hide_banner', '-nostats', '-v', 'panic', '-i', '-', '-codec:a', 'libmp3lame', '-ar', '48000', '-b:a', '192k', '-vn', '-f', 'mp3', 'Jon Bellion - All Time Low.mp3.temp']
6%|██████▋ | 208/3712KiB [00:00<00:12, 281.23KiB/s]DEBUG:spotdl.lyrics.providers.genius:130:
Fetching lyric text from "https://genius.com/Jon-bellion-all-time-low-lyrics".
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3712/3712KiB [00:15<00:00, 237.00KiB/s]
INFO:spotdl.command_line.core:293:
Applying metadata
DEBUG:spotdl.metadata.embedders.default_embedder:53:
Writing MP3 metadata to "Jon Bellion - All Time Low.mp3.temp".
DEBUG:spotdl.command_line.core:286:
Renaming Jon Bellion - All Time Low.mp3.temp to Jon Bellion - All Time Low.mp3.
DEBUG:spotdl.authorize.services.spotify:32:
Reading cached master Spotify credentials.
INFO:spotdl.command_line.core:344:
2. Downloading "https://open.spotify.com/track/5EeXZLPDNCWLpEHzNcL1U7"
DEBUG:spotdl.metadata_search:182:
Extracting YouTube and Spotify metadata for input Spotify URI.
DEBUG:spotdl.metadata_search:248:
Extracting Spotify metadata for input Spotify URI.
DEBUG:spotdl.metadata.providers.spotify:25:
Fetching Spotify metadata for "https://open.spotify.com/track/5EeXZLPDNCWLpEHzNcL1U7".
DEBUG:spotdl.metadata_search:248:
Extracting Spotify metadata for input Spotify URI.
DEBUG:spotdl.metadata.providers.spotify:25:
Fetching Spotify metadata for "https://open.spotify.com/track/5EeXZLPDNCWLpEHzNcL1U7".
DEBUG:spotdl.metadata.providers.youtube:119:
Fetching YouTube results for "Lindsey Stirling - Hold My Heart (feat. ZZ Ward) lyrics" at "https://www.youtube.com/results?sp=EgIQAQ%253D%253D&q=Lindsey%20Stirling%20-%20Hold%20My%20Heart%20%28feat.%20ZZ%20Ward%29%20lyrics".
DEBUG:spotdl.metadata.providers.youtube:43:
Matched with: Lindsey Stirling - Hold My Heart feat. ZZ Ward - Lyric Video (https://www.youtube.com/watch?v=Ygv4T0bH2Sc) [3:31]
DEBUG:spotdl.metadata.providers.youtube:225:
Fetching YouTube metadata for "https://www.youtube.com/watch?v=Ygv4T0bH2Sc".
DEBUG:spotdl.metadata.providers.youtube:236:
YouTube returned an unexpected response for `pytube.YouTube(https://www.youtube.com/watch?v=Ygv4T0bH2Sc)`. Retries left: 4
DEBUG:spotdl.metadata.providers.youtube:225:
Fetching YouTube metadata for "https://www.youtube.com/watch?v=Ygv4T0bH2Sc".
DEBUG:spotdl.metadata.providers.youtube:236:
YouTube returned an unexpected response for `pytube.YouTube(https://www.youtube.com/watch?v=Ygv4T0bH2Sc)`. Retries left: 3
DEBUG:spotdl.metadata.providers.youtube:225:
Fetching YouTube metadata for "https://www.youtube.com/watch?v=Ygv4T0bH2Sc".
DEBUG:spotdl.metadata.providers.youtube:236:
YouTube returned an unexpected response for `pytube.YouTube(https://www.youtube.com/watch?v=Ygv4T0bH2Sc)`. Retries left: 2
DEBUG:spotdl.metadata.providers.youtube:225:
Fetching YouTube metadata for "https://www.youtube.com/watch?v=Ygv4T0bH2Sc".
DEBUG:spotdl.metadata.providers.youtube:236:
YouTube returned an unexpected response for `pytube.YouTube(https://www.youtube.com/watch?v=Ygv4T0bH2Sc)`. Retries left: 1
DEBUG:spotdl.metadata.providers.youtube:225:
Fetching YouTube metadata for "https://www.youtube.com/watch?v=Ygv4T0bH2Sc".
DEBUG:spotdl.metadata.providers.youtube:236:
YouTube returned an unexpected response for `pytube.YouTube(https://www.youtube.com/watch?v=Ygv4T0bH2Sc)`. Retries left: 0
DEBUG:spotdl.metadata.providers.youtube:225:
Fetching YouTube metadata for "https://www.youtube.com/watch?v=Ygv4T0bH2Sc".
Traceback (most recent call last):
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/extract.py", line 297, in apply_descrambler
for format_item in formats
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/extract.py", line 297, in <listcomp>
for format_item in formats
KeyError: 'url'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/spotdl/metadata/providers/youtube.py", line 227, in from_url
content = pytube.YouTube(url)
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/__main__.py", line 92, in __init__
self.descramble()
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/__main__.py", line 132, in descramble
apply_descrambler(self.player_config_args, fmt)
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/extract.py", line 301, in apply_descrambler
parse_qs(formats[i]["cipher"]) for i, data in enumerate(formats)
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/extract.py", line 301, in <listcomp>
parse_qs(formats[i]["cipher"]) for i, data in enumerate(formats)
KeyError: 'cipher'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/extract.py", line 297, in apply_descrambler
for format_item in formats
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/extract.py", line 297, in <listcomp>
for format_item in formats
KeyError: 'url'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/spotdl/metadata/providers/youtube.py", line 227, in from_url
content = pytube.YouTube(url)
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/__main__.py", line 92, in __init__
self.descramble()
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/__main__.py", line 132, in descramble
apply_descrambler(self.player_config_args, fmt)
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/extract.py", line 301, in apply_descrambler
parse_qs(formats[i]["cipher"]) for i, data in enumerate(formats)
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/extract.py", line 301, in <listcomp>
parse_qs(formats[i]["cipher"]) for i, data in enumerate(formats)
KeyError: 'cipher'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/extract.py", line 297, in apply_descrambler
for format_item in formats
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/extract.py", line 297, in <listcomp>
for format_item in formats
KeyError: 'url'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/spotdl/metadata/providers/youtube.py", line 227, in from_url
content = pytube.YouTube(url)
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/__main__.py", line 92, in __init__
self.descramble()
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/__main__.py", line 132, in descramble
apply_descrambler(self.player_config_args, fmt)
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/extract.py", line 301, in apply_descrambler
parse_qs(formats[i]["cipher"]) for i, data in enumerate(formats)
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/extract.py", line 301, in <listcomp>
parse_qs(formats[i]["cipher"]) for i, data in enumerate(formats)
KeyError: 'cipher'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/extract.py", line 297, in apply_descrambler
for format_item in formats
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/extract.py", line 297, in <listcomp>
for format_item in formats
KeyError: 'url'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/spotdl/metadata/providers/youtube.py", line 227, in from_url
content = pytube.YouTube(url)
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/__main__.py", line 92, in __init__
self.descramble()
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/__main__.py", line 132, in descramble
apply_descrambler(self.player_config_args, fmt)
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/extract.py", line 301, in apply_descrambler
parse_qs(formats[i]["cipher"]) for i, data in enumerate(formats)
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/extract.py", line 301, in <listcomp>
parse_qs(formats[i]["cipher"]) for i, data in enumerate(formats)
KeyError: 'cipher'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/extract.py", line 297, in apply_descrambler
for format_item in formats
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/extract.py", line 297, in <listcomp>
for format_item in formats
KeyError: 'url'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/spotdl/metadata/providers/youtube.py", line 227, in from_url
content = pytube.YouTube(url)
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/__main__.py", line 92, in __init__
self.descramble()
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/__main__.py", line 132, in descramble
apply_descrambler(self.player_config_args, fmt)
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/extract.py", line 301, in apply_descrambler
parse_qs(formats[i]["cipher"]) for i, data in enumerate(formats)
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/extract.py", line 301, in <listcomp>
parse_qs(formats[i]["cipher"]) for i, data in enumerate(formats)
KeyError: 'cipher'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/extract.py", line 297, in apply_descrambler
for format_item in formats
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/extract.py", line 297, in <listcomp>
for format_item in formats
KeyError: 'url'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/eeffeemorgan/.local/bin/spotdl", line 10, in <module>
sys.exit(main())
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/spotdl/command_line/__main__.py", line 48, in main
spotdl.match_arguments()
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/spotdl/command_line/core.py", line 79, in match_arguments
self.arguments["list"],
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/spotdl/command_line/core.py", line 346, in download_tracks_from_file
metadata = search_metadata.on_youtube_and_spotify()
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/spotdl/metadata_search.py", line 89, in on_youtube_and_spotify
metadata = caller()
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/spotdl/metadata_search.py", line 186, in _on_youtube_and_spotify_for_type_spotify
youtube_metadata = self.providers["youtube"].from_url(youtube_video["url"])
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/spotdl/metadata/providers/youtube.py", line 239, in from_url
return self.from_url(url, retries=retries)
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/spotdl/metadata/providers/youtube.py", line 239, in from_url
return self.from_url(url, retries=retries)
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/spotdl/metadata/providers/youtube.py", line 239, in from_url
return self.from_url(url, retries=retries)
[Previous line repeated 2 more times]
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/spotdl/metadata/providers/youtube.py", line 227, in from_url
content = pytube.YouTube(url)
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/__main__.py", line 92, in __init__
self.descramble()
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/__main__.py", line 132, in descramble
apply_descrambler(self.player_config_args, fmt)
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/extract.py", line 301, in apply_descrambler
parse_qs(formats[i]["cipher"]) for i, data in enumerate(formats)
File "/home/eeffeemorgan/.local/lib/python3.7/site-packages/pytube/extract.py", line 301, in <listcomp>
parse_qs(formats[i]["cipher"]) for i, data in enumerate(formats)
KeyError: 'cipher'
bug
eeffeemorgan
All 6 comments
Not a solution, but this is caused by nficano/pytube#641. Seeing as that issue popped up only months after the latest commit on that repo, it's probably because something happened on the youtube end, idrk.
TheBrenny
on 29 May 2020
🚀1
B4RN3SCODE
on 30 May 2020
As @TheBrenny reported in https://github.com/ritiek/spotify-downloader/issues/737#issuecomment-635732391, the issue is on pytube's side, and since pytube is one of our dependencies, we're being affected. There's already a PR open in pytube that fixes this issue but it seems like pytube's maintainers aren't active at the moment and so it hasn't been merged into the main tree yet.
I'll wait for another day and if by then pytube still doesn't merge the fix and publish a new release, I'll apply the patch to pytube here in spotdl itself and publish our own new release.
ritiek
on 30 May 2020
👍3
Ok, pytube still hasn't merged the fix so I've released spotdl v2.0.8 which will internally patch pytube to fix the problem. Update your spotdl with:
$ pip3 install spotdl -U
and downloads should work now.
ritiek
on 31 May 2020
👍2
Just a note for anyone using the new release, I had to uninstall and re-install spotdl for it to work.
Otherwise I had a error of spodtl.patch
LolnationCH
on 31 May 2020
Just go to the pytubeextract.py (in pytube library)file: the path of file will be : "C:ProgramDataAnaconda3libsite-packagespytubeextract.py", (in windows)
Open extract,py file and search for line: parse_qs(formats[i]["cipher"]) for i, data in enumerate(formats)
now replace 'cipher' with 'signatureCipher'.
save it.
now run your code again
MukulKirtiVerma
on 21 Jul 2020
Was this page helpful?
0 / 5 - 0 ratings
Related issues
marinabar
·
4Comments
Maxino22
·
4Comments
noahball
·
3Comments
ritiek
·
5Comments
MikhailZex
·
5Comments
Most helpful comment
As @TheBrenny reported in https://github.com/ritiek/spotify-downloader/issues/737#issuecomment-635732391, the issue is on pytube's side, and since pytube is one of our dependencies, we're being affected. There's already a PR open in pytube that fixes this issue but it seems like pytube's maintainers aren't active at the moment and so it hasn't been merged into the main tree yet.
I'll wait for another day and if by then pytube still doesn't merge the fix and publish a new release, I'll apply the patch to pytube here in spotdl itself and publish our own new release.