Jackett: [AvistaZ | PrivateHD | CinemaZ] Verification required (Config) [Solved - Upgrade to 0.16.526 & above]

[RoloSoze]

Environment

OS: Windows 10 Pro 64-bit

.Net Runtime: N/A

.Net Version: N/A

Jackett Version: 0.16.515.0

Last Working Jackett Version: 0.16.515.0

Are you using a proxy or VPN? No

Description

I restarted my router a few times earlier today to test the changes made to resolve other issues (#6876 & #8710). It seems that the 3 AvistaZ Network trackers did not like that, so now I'm unable to configure any of them. On their respective websites, I was signed out and had to sign in and solve the "I'm not a robot" CAPTCHA, but Jackett does not provide this option. I guess this is the issue? I also tried removing the indexers and adding them back, but that did not work either.

EDIT: It seems the issue is affecting all users of the AvistaZ Network trackers. AnimeTorrents (AKA AnimeZ) is working fine; not sure about ExoticaZ since I'm not a member.

Logged Error Messages

I get the same error when trying to configure all 3 trackers:

An error occurred while updating this indexer
Verification required

Enhanced log (for PrivateHD, but I assume it's the same for all 3):
log.txt

Screenshots

Not needed

[Curtis-L]

i met the same issue miniutes ago, the avz alliance sites have google recaptcha enabled on its login page, i guess jackett would need to switch to cookie for them.

unfortunately their cookie become invalid randomly, this means they would need to be reconfigured from time to time.

(sigh

[RoloSoze]

@Curtis-L Oh I thought it was only happening to me because of my repeated IP changes earlier today. Apparently it's happening to everyone then. As for their cookies, I think they expire a couple of times a month or so? So it shouldn't be that bad, but yeah having to reconfigure them every now and then is not ideal.

Hopefully @ngosang gets the chance to work on this soon. He might be facing the same issue.

[GetsTheStuffs]

Same problem for me on PrivateHD, so yeah definitely an issue to be addressed.

[0-MegaMind-0]

I think they expire a couple of times a month or so

they're set to one month by default but the site will logout everyone when there is a update to the site

So yes when this happens you'll need to manually update them

[RoloSoze]

They're set to one month by default but the site will logout everyone when there is a update to the site

So yes when this happens you'll need to manually update them

Thank you for the clarification. Yeah I suppose once a month is not bad. We'll have to live with it.

[0-MegaMind-0]

Thank you for the clarification. Yeah I suppose once a month is not bad. We'll have to live with it.

but it wont matter on Jackett since modifying the cookie expire manually works so basically the only thing that'll effect someone is the site updates and there's been few of them recently and when it happens it happens multiple times in a single day

Hopefully we'll see a API sometime in the future but till then this is the only solution

[ngosang]

@RoloSoze Could you talk with the staff to allow searching with passkey/pid? I prefer to avoid the cookie method. I have been playing with the RSS feeds but they don't allow text search.
Eg: https://privatehd.to/rss/feed?fid=14422&pid=3fe5cc3b8b42661e0c8e3cacda

They should change the API to allow:
privatehd.to/torrents?in=1&search=harry&type=0 &pid=3fe5cc3b8b42661e0c8e3cacda

[Curtis-L]

@0-MegaMind-0 @RoloSoze no not once a month, i use my pricate and dedicated vpn to browse them, only the avz sites' cookie expire even within just a couple of days, i guess this have to do with the site update and reset, they do this very often and i even turned off the 2fa because i needed to relogin too many times. besides, jackett doesn't support them with 2fa.

[kilroyma]

Having the same issue, so what exactly do I need to do to fix it?

[Curtis-L]

@kilroyma there's nothing you can do, wait until jackett release a new version.

[abixalmon]

I will be adding token based login for all AvistaZ sites, please dont hack login!

I will also make API available for search and torrent listing

[abixalmon]

Can someone (Jackett Developer) give me list of requirements, so i can push the Json API ASAP.

[ngosang]

Take a look at this code => https://github.com/Jackett/Jackett/blob/85b34ffe1f3961b7cb4c0469dd5cc87c3008a8b3/src/Jackett.Common/Indexers/SceneHD.cs

Some requirements:

• All API endpoints must work without cookies (passkey query parameter or header with passkey)
• All searches must return 100 results
• All searches must support category filter (several categories related to file type + quality/resolution)
• Empty search => returns newest/latests torrents
• Text search => search text in the title (ordered by date or configurable)
• IMDB search => search imdb id
• For each result include all required fields, banner and download link (see the code above).

I think the best is you make a first version and I will ask you for changes.

[RoloSoze]

@RoloSoze Could you talk with the staff to allow searching with passkey/pid?

@ngosang Brought the boss here :) Thanks for helping him with the requirements.

[abixalmon]

Authentication Request

curl --location --request POST 'https://privatehd.to/api/v1/jackett/auth' \
--header 'Accept: application/json' \
--header 'Content-Type: application/json' \
--form 'username=USERNAME' \
--form 'password=PASSOWORD' \
--form 'pid=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa'

This will return Auth Token on successful request

{
    "token": "RaNDomStringAqUicKBroWnFoxJumpSoVerTheLaZydOgRand",
    "expiry": 86388
}

or errors on failure

Form Errors
{
    "username": [
        "The username field is required."
    ],
    "password": [
        "The password field is required."
    ],
    "pid": [
        "The pid field is required."
    ]
}

Invalid Login Error
{
    "message": "Login Failed: Invalid username, password or PID"
}

Banned Users
{
    "message": "You dont have download access"
}

Low Ranked Users (Leech)
{
    "message": "Only member rank and above can use Jackett"
}

PID Can be found on your profile page:

[abixalmon]

Querying Torrents Page

curl --location --request GET 'https://privatehd.to/api/v1/jackett/torrents' \
--header 'Authorization: Bearer RaNDomStringAqUicKBroWnFoxJumpSoVerTheLaZydOgRand'

You must use Auth Token with every request. (token generated in previous post)
Auth Token once generated expires in 24 hours, unless revoked by an admin.

Errors

// If no token is provided.
401: Unauthenticated.

// If token fails to validate,
401: Authentication Failed. Invalid Token

// If token is expired
412: Authentication Failed. Token Expired

This will return 100 latest torrents

{
    "current_page": 1,
    "data": [
        {
            "id": 139625,
            "url": "https://privatehd.to/torrent/139625-rugal-2020-complete-1080p-nf-web-dl-ddp20-x264-felicityfornow",
            "download": "https://privatehd.to/api/v1/jackett/download/139625/139625-45454878745645878564.torrent",
            "type_id": 2,
            "type": "TV-Show",
            "info_hash": "d2ec16de44b27321713dd128806489b4c8323794",
            "file_name": "Rugal (2020) Complete 1080p NF WEB-DL DDP2.0 x264 - FelicityForNow",
            "file_size": 34021683830,
            "file_count": 16,
            "seed": 0,
            "leech": 13,
            "completed": 0,
            "downloaded": 17,
            "upload_multiply": 1,  // 1 for normal, 2 for double upload
            "download_multiply": 0, // 1 for normal, 0 for Freeleech, 0.5 for half-download
            "adult": false,
            "video_quality_id": 3,
            "video_quality": "1080p",
            "rip_type_id": 12,
            "rip_type": "WEB-DL",
            "movie_tv": {
                "imdb": "tt11769798",
                "tmdb": "97766",
                "tvdb": "372053",
                "title": "Rugal",
                "tv_season": null,
                "tv_episode": null,
                "tv_full_season": false,
                "tv_complete": true
            },
            "audio": [
                {
                    "id": 91,
                    "language": "Korean"
                }
            ],
            "subtitle": [
                {
                    "id": 33,
                    "language": "Chinese"
                },
                {
                    "id": 45,
                    "language": "English"
                },
                {
                    "id": 72,
                    "language": "Indonesian"
                },
                {
                    "id": 91,
                    "language": "Korean"
                },
                {
                    "id": 131,
                    "language": "Portuguese"
                },
                {
                    "id": 151,
                    "language": "Spanish"
                },
                {
                    "id": 162,
                    "language": "Thai"
                },
                {
                    "id": 176,
                    "language": "Vietnamese"
                }
            ],
            "music_type": null,
            "created_at": "2020-05-20 09:48:29"
        },
        ...,
        ...,
    "from": 1,
    "last_page": 649,
    "next_page_url": "https://privatehd.to/api/v1/jackett/torrents?page=2",
    "path": "https://privatehd.to/api/v1/jackett/torrents",
    "per_page": 100,
    "prev_page_url": null,
    "to": 100,
    "total": 64805
}

For querying and filtering, you can pass every parameters that works on our torrents page: https://privatehd.to/torrents.

curl --location --request GET 'https://privatehd.to/api/v1/jackett/torrents?type=1&rip_type%5B%5D=14&discount%5B%5D=3' \
--header 'Authorization: Bearer RaNDomStringAqUicKBroWnFoxJumpSoVerTheLaZydOgRand'

You can also search by imdb, tmdb or tvdb IDs

https://privatehd.to/api/v1/jackett/torrents?imdb=tt123456

https://privatehd.to/api/v1/jackett/torrents?tmdb=1234

https://privatehd.to/api/v1/jackett/torrents?tvdb=3653

[RoloSoze]

@ngosang You may want to check later whether the API addresses the issues we had here https://github.com/Jackett/Jackett/pull/8050 for the SD categories and episodes without seasons.

[ngosang]

@abixalmon Thank you, I will work on it later today.

1. Is this already available in all AvistaZ sites?

1. Why we need the user + pass + pid? It looks redundant to me. Could you use user+pass OR pid?

2. Related to 2) A token that last forever is a security hole. If Jackett is compromised I can't revoke the token. Wouldn't it be better to send the pid directly in the header? The user can request a new one. If the PID is wrong (invalid, banned, low rank...) return the error, I will show it to the user.

3. Maybe I will request some changes in categories, but it's ok for now.

[abixalmon]

@abixalmon Thank you, I will work on it later today.

1. Is this already available in all AvistaZ sites?

Only enabled on PrivateHD for now. will be available soon on all sites.

1. Why we need the user + pass + pid? It looks redundant to me. Could you use user+pass OR pid?

So, we can limit bots trying to guess username and password, This was the main reason we had to add captcha on login page, we were getting 1000s of requests trying to login using random combination of username and password using Random IPs.

1. Related to 2) A token that last forever is a security hole. If Jackett is compromised I can't revoke the token. Wouldn't it be better to send the pid directly in the header? The user can request a new one. If the PID is wrong (invalid, banned, low rank...) return the error, I will show it to the user.

That makes sense, I will add expiry time to token, maybe 24 hours?

1. Maybe I will request some changes in categories, but it's ok for now.

I have updated the code and added comments on my previous post!

[ngosang]

So, we can limit bots trying to guess username and password, This was the main reason we had to add captcha on login page, we were getting 1000s of requests trying to login using random combination of username and password using Random IPs.

If we use only the pid we don't have this issue neither. It's impossible to guess a long hash... In the RSS feeds the user+pass is not included in the URL, only the pid.

That makes sense, I will add expiry time to token, maybe 24 hours?

Good, Jackett will renew the token automatically.

[0-MegaMind-0]

If we use only the pid we don't have this issue neither. It's impossible to guess a long hash... In the RSS feeds the user+pass is not included in the URL, only the pid.

Problem is we had this issue with rss with people posting their PID everywhere but abi did introduce a ACL which needs to enabled on user account settings for RSS to now function

but this isn't bad either but only PID will be more convenient for sure

For everything posted above I tested some of it and it works
But do note that currently external subtitle will be listed in the json output

[RoloSoze]

For everything posted above I tested some of it and it works

@0-MegaMind-0 The only issue we had previously with any of the AvistaZ Network trackers is with AvistaZ's SD categories and episodes without seasons (#8050). Initially Jackett did not return all the results from these categories but @ngosang fixed it. Once the API is ready for AZ, you guys might want to make sure that it covers those categories too.

[abixalmon]

Enabled on AvistaZ

[ngosang]

@abixalmon I have the Jackett integration almost ready but there are some remaining issues.

I have not implemented a very smart mechanism for token management. Some users may request more than 1 token per day. I hope this is not a problem.

The AvistaZ, PrivateHD and CinemaZ code is coupled in Jackett. I can't release a fix for Jackett until the changes are deployed on all 3 sites.

What about ExoticaZ and AnimeZ? We have to use the API too?

Remaining issues:

Authorization

• Please, change the HTTP response code in the Auth Endpoint for bad authorization to 401. Currently you are returning 403. It's right but is causing us troubles (we have a 3rd party library to bypass Cloudflare that uses 403 code to detect challenges...)

• (Optional) Change the login error messages to unify the response format. This is simplifies our code:

{
    "username": [
        "The username field is required."
    ],
    "password": [
        "The password field is required."
    ],
    "pid": [
        "The pid field is required."
    ]
}

to

{
    "message": "The username field is required."
}

Search

• Imdb search is not working as expected if there aren't results. Try tt11041045 for example. You are returning newest torrents instead of an empty list.

• Change download link with real link.

• (optional) add number of files

• (optional) add banner/cover image url. eg: https://privatehd.to/images/posters/s/0/5/s053ligghrah.mid.jpg

[Hendo-AU]

Also having this issue, Is there a way for me to fix it or do I just have to wait for a docker update?

[abixalmon]

Form Error will return Error 422

{
    "message": "The username field is required."
}

Rest of Auth will return error 401

{
    "message": "Login Failed: Invalid username, password or PID"
}

External ID Search will return empty result set if not found

{
    "current_page": 1,
    "data": [],
    "from": null,
    "last_page": 0,
    "next_page_url": null,
    "path": "https://privatehd.to/api/v1/jackett/torrents",
    "per_page": 100,
    "prev_page_url": null,
    "to": null,
    "total": 0
}

Added File Count

{
...
"data": [
...
     "file_size": 34021683830,
     "file_count": 16,
...
]
...
}

Even if i enable images it wont work, as we have hot-linking blocked, I would suggest you query image on TheMovieDB or OMDB using imdb or tmdb ID.

[abixalmon]

For Torrent Downloads to work, users must have RSS enabled on their "Account Setting"

[abixalmon]

Fixed upload_multiply and download_multiply.
It will show 1 for normal torrents (no discounts)

[abixalmon]

Updated Response

https://avistaz.to/api/v1/jackett/torrents?limit=1

{
    "current_page": 1,
    "data": [
        {
            "id": 139625,
            "url": "http://avistaz.abit/torrent/139625-rugal-2020-complete-1080p-nf-web-dl-ddp20-x264-felicityfornow",
            "download": "http://avistaz.abit/rss/download/aaaaaaaaaaaaaaaaaaaaaaaa/1111-ababababababababaab.torrent",
            "type_id": 2,
            "type": "TV-Show",
            "info_hash": "d2ec16de44b27321713dd128806489b4c8323794",
            "file_name": "Rugal (2020) Complete 1080p NF WEB-DL DDP2.0 x264 - FelicityForNow",
            "file_size": 34021683830,
            "file_count": 16,
            "seed": 0,
            "leech": 13,
            "completed": 0,
            "downloaded": 17,
            "upload_multiply": 1,
            "download_multiply": 0,
            "adult": false,
            "video_quality_id": 3,
            "video_quality": "1080p",
            "rip_type_id": 12,
            "rip_type": "WEB-DL",
            "movie_tv": {
                "imdb": "tt11769798",
                "tmdb": "97766",
                "tvdb": "372053",
                "title": "Rugal",
                "tv_season": null,
                "tv_episode": null,
                "tv_full_season": false,
                "tv_complete": true
            },
            "audio": [
                {
                    "id": 91,
                    "language": "Korean"
                }
            ],
            "subtitle": [
                {
                    "id": 33,
                    "language": "Chinese"
                },
                {
                    "id": 45,
                    "language": "English"
                },
                {
                    "id": 72,
                    "language": "Indonesian"
                },
                {
                    "id": 91,
                    "language": "Korean"
                },
                {
                    "id": 131,
                    "language": "Portuguese"
                },
                {
                    "id": 151,
                    "language": "Spanish"
                },
                {
                    "id": 162,
                    "language": "Thai"
                },
                {
                    "id": 176,
                    "language": "Vietnamese"
                }
            ],
            "music_type": null,
            "created_at": "2020-05-20 09:48:29"
        }
    ],
    "from": 1,
    "last_page": 64805,
    "next_page_url": "http://avistaz.abit/api/v1/jackett/torrents?page=2",
    "path": "http://avistaz.abit/api/v1/jackett/torrents",
    "per_page": "1",
    "prev_page_url": null,
    "to": 1,
    "total": 64805
}

[RoloSoze]

@ngosang @abixalmon Thank you guys for working on this issue in such a timely manner. Your efforts are sincerely appreciated.

@garfield69 Since all users of these 3 trackers are affected by and waiting for this issue to be resolved, may I suggest making an exception and releasing an update ASAP to fix this instead of waiting for the usual daily update?

[ngosang]

@abixalmon Thank you!

AvistaZ, PrivateHD and CinemaZ are fixed in Jackett v0.16.526
You have to configure the indexers in Jackett again because the PID is now required. Read the notes in the configuration!!!
If you find any problem, please, report in this issue.

[BogartBme]

Being new to all this, I have to ask, If I'm using Jackett via unraid docker, I'm assuming I have to wait for the updated docker. Is that correct?

[mrjackyliang]

Will wait for the linuxserver.io latest tag to update itself.

[sfkpmr]

@ngosang PID should probably be hidden like passwords.

[jbroo144]

I enabled RSS feeds, updated Jackett to 0.16.526.0 and added the PID, yet it was returning "Login Failed: Invalid username, password or PID".

It turns out that my previous username in Jackett of "[email protected]" was being rejected. Once I swapped to "username" without "@outlook.com", it was accepted and everything works.

I found that interesting because firstly, the "@outlook.com" username worked before implementing captcha on PrivateHD and secondly, the login username for PrivateHD includes "@outlook.com".

Thought I would mention this in case someone else encounters this issue.

[0-MegaMind-0]

@jbroo144 api does support email as username at the moment
before it used to use direct browser login which has the email / username field so passing email would have worked fine

[ngosang]

PID should probably be hidden like passwords.

Sorry, it's not possible with the current code base. This could be a security issue but it's not critical because you need the password too. The password is hidden and encrypted.

Being new to all this, I have to ask, If I'm using Jackett via unraid docker, I'm assuming I have to wait for the updated docker. Is that correct?
Will wait for the linuxserver.io latest tag to update itself.

I'm going to talk with linuxserver so you can receive latest updates without pulling the new Docker image.

[BogartBme]

I applied the new updated docker to Jackett 16.527 and now get a different error
An error occurred while testing this indexer
Exception (privatehd): Object reference not set to an instance of an object.: Object reference not set to an instance of an object.

Edit: I just noticed that I had to enter in the PID. Once I did that it worked successfully.

[garfield69]

@abixalmon FYI with the switch to the API, and with my particular PrivateHD account setting (download/upload is disabled), I am unable to login using the Jackett PrivateHD indexer, as the login is rejected with You dont have download access.
I am still able to use the browser to access the site.