Newpipe: Youtube import subscription instructions is not working anymore

[simula67]

Description

Following the instructions at https://newpipe.schabi.org/FAQ/tutorials/import-export-data/#import-youtube to download subscriptions no longer works as it seems like Youtube has changed its interface. Going to https://www.youtube.com/subscription_manager?action_takeout=1 no longer starts a download, but it redirects to https://www.youtube.com/feed/channels. This prevents importing of subscriptions list which could have been then loaded to Newpipe app.

Expected behavior

It should be possible to import subscriptions from youtube account using instructions at https://newpipe.schabi.org/FAQ/tutorials/import-export-data/#import-youtube.

Actual Behavior

Instructions given to import subscriptions from Youtube not working anymore. The instructions ask you to download subscriptions using the URL: https://www.youtube.com/subscription_manager?action_takeout=1, but this merely redirects to https://www.youtube.com/feed/channels and does not start a download.

Steps to reproduce

1. Read import instructions from https://newpipe.schabi.org/FAQ/tutorials/import-export-data/#import-youtube

2. Follow the link at https://www.youtube.com/subscription_manager?action_takeout=1

3. Gets redirected to https://www.youtube.com/feed/channels. No download starts.

4. Cannot import subscriptions from youtube

Platform

All platforms

[Stypox]

Could you provide the new link youtube uses to download subscriptions?

[opusforlife2]

@Stypox https://github.com/TeamNewPipe/NewPipe/issues/1751#issuecomment-718813324

[simula67]

@Stypox

Seems like a multi-step process now:

1. Go to something like : https://takeout.google.com/takeout/custom/youtube?continue=https%3A%2F%2Fmyaccount.google.com%2Fyourdata%2Fyoutube%3Fhl%3Den_GB%26authuser%3D0

2. Click on "All data included"

3. Select "Deselect all" and select only "subscriptions" and click on OK

4. Go to "Next step"

5. Click "Create export"

6. Download link will be sent to email by default

7. Use the emailed link to download to obtain subscriptions.json

[opusforlife2]

Download link will be sent to email by default

When you say "by default", does that mean there are other options?

[simula67]

Yes there are other options like "Add to Drive", "Add to Dropbox", "Add to OneDrive" and "Add to Box" besides the default delivery method, which is "Send download link via email".

[inigosola]

@Stypox

Seems like a multi-step process now:

1. Go to something like : https://takeout.google.com/takeout/custom/youtube?continue=https%3A%2F%2Fmyaccount.google.com%2Fyourdata%2Fyoutube%3Fhl%3Den_GB%26authuser%3D0
2. Click on "All data included"
3. Select "Deselect all" and select only "subscriptions" and click on OK
4. Go to "Next step"
5. Click "Create export"
6. Download link will be sent to email by default
7. Use the emailed link to download to obtain subscriptions.json

I have followed these steps, I get a subscriptions.json file, but when I import the file into NewPipe, I get an error and it doesn't work.

[Djay86]

I also tried importing the .json to newpipe but it doesn't know what to do with it unless there is some way to convert it for newpipe.

[Stypox]

@simula67 great, thank you. I will try to implement this

[Saecki]

I've converted my subscriptions.json file in a quite naive shell script to a newpipe fomat. You'll need jq and ripgrep but if you just want a quick and dirty fix:

subscriptions=$(cat $1 | jq '.[]' | jq '.snippet.title,.snippet.resourceId.channelId' | rg --multiline --no-filename -e '"(.*?)"\n"(.*?)"\n' -r '{
    "service_id": 0,
    "url": "https://www.youtube.com/channel/$2",
    "name": "$1"
},')

json="{
\"app_version\": \"0.20.2\",
\"app_version_int\": 956,
\"subscriptions\": [
${subscriptions::-1}
]
}"

echo "$json" > newpipe_subscriptions.json

[IvanProgramming]

I've converted my subscriptions.json file in a quite naive shell script to a newpipe fomat. You'll need jq and ripgrep but if you just want a quick and dirty fix:

subscriptions=$(cat $1 | jq '.[]' | jq '.snippet.title,.snippet.resourceId.channelId' | rg --multiline --no-filename -e '"(.*?)"\n"(.*?)"\n' -r '{
    "service_id": 0,
    "url": "https://www.youtube.com/channel/$2",
    "name": "$1"
},')

json="{
\"app_version\": \"0.20.2\",
\"app_version_int\": 956,
\"subscriptions\": [
${subscriptions::-1}
]
}"

echo "$json" > newpipe_subscriptions.json

Sorry for some dummy question, but can you give an example of usage this script in bash command line?

[Saecki]

So basically you could create a convert.sh file and copy the entire script in there.

#!/bin/sh
subscriptions=$(cat $1 | jq '.[]' | jq '.snippet.title,.snippet.resourceId.channelId' | rg --multiline --no-filename -e '"(.*?)"\n"(.*?)"\n' -r '{
    "service_id": 0,
    "url": "https://www.youtube.com/channel/$2",
    "name": "$1"
},')

json="{
\"app_version\": \"0.20.2\",
\"app_version_int\": 956,
\"subscriptions\": [
${subscriptions::-1}
]
}"

echo "$json" > $2

To make it executable:
chmod +x convert.sh
Then it can be used like:
./convert.sh <input_file> <output_file>
In this case:
./convert.sh subscriptions.json newpipe_subscriptions.json

[IvanProgramming]

@Saecki Thank You for such detailed description. And excuse for some troubling

[Stypox]

Oh no. I just tried the steps you provided above and everything seems to be localized, even the path inside the .zip file provided... We would have to ask the user to extract the json file themselves, making everything really complicated

[Saecki]

Maybe it's easier to parse the website after all.

[Stypox]

Could you all test the apk provided in #4759

[simula67]

The apk works correctly to import subscriptions. Thanks for doing this.

[Ammako]

@Saecki

./convert.sh: 14: ./convert.sh: Bad substitution

[rozenj]

@Ammako
One-liner using only jq

jq '{app_version: "0.20.2", app_version_int: 956, subscriptions: [.[] | {service_id: 0, url: ("https://www.youtube.com/channel/" + .snippet.resourceId.channelId), name: .snippet.title }]}' [subscriptions filename here] > newpipe_subscriptions.json