Jackett: [Enhancement] add logic behind plus button - detect if indexer is using cookie method - invoke wrench action

[ilike2burnthing]

Testing here done with v0.14.275, as well as v0.14.222 and v0.14.243 to confirm when change happened.

Tested with both current Docker container and clean Windows sandbox install.

I'm guessing this is as a result of https://github.com/Jackett/Jackett/commit/c78cd997ee1afa50c56f8ccc98c6eb68ec2f6d8b and https://github.com/Jackett/Jackett/commit/ef73523c8ee1ff87e0910c7abf5bb2ef94d683fd .

To reproduce:

1. remove Demonoid and/or TorrentDownload if you have them configured
2. disconnect/block from internet, alternatively use fake proxy details or remove logon details from real proxy
3. click Add Indexer
4. click + for Demonoid and/or TorrentDownload
5. get error and no new indexer in configured list

I tested all public indexers, the majority were added successfully, except for:

• Anidex
• Anime Tosho
• Corsaro.red
• Demonoid
• DivxTotal
• EpubLibre
• Horrible Subs
• Internet Archive
• MejorTorrent
• Newpct
• RARBG
• ShowRSS
• Solid Torrents
• TorrentDownload
• TorrentGalaxy.org
• Torrents.csv
• YTS

The rest of these fail in v0.14.222 as well, only Demonoid and TorrentDownload developed this behaviour with v0.14.243. That these are in the minority, I assume it's not the intended result. If there were a temporary issue with a site or for a user, they would be unable to add the indexer at all.

In some cases clicking the configuration wrench can help, e.g. choosing a proxy, adding a cookie, or entering a Captcha code, but those aren't options for most of the indexers listed.

I would assume this also affects some private and semi-private trackers, but I'm unable to test them.

[garfield69]

Yes. Any indexer that uses the cookie method will, if the config is added the first time, need the cookie to be loaded in the config for the indexer to pass the test and be added to the active page.
The plus button should act like the wrench button for these indexes.
We may need to add logic behind the plus button if possible to detect if the indexer is using the cookie method and invoke the wrench anction instead of attempting to add the indexer to the active page.

[ilike2burnthing]

From what I can tell, the majority of indexers (public ones at least) aren't being tested before being added. Even with the internet disconnected, or blocked by fake proxy, I was able to add all but that list above.

[garfield69]

most public indexer do not have a login block in the yaml so there is no testing required and get them added to the active list.
but public indexer that have the login block (to support the cookie method session because the respective sites are using cloudflare or other challenges that prevent straight session access to the sites browser page) need to pass the test before they get added to the active list, hence the cause of the problem you have raised.

[ilike2burnthing]

Ah ok! Gotcha now.

So that explains Demonoid and TorrentDownload, they have the cookie method, and TGx, which has the Captcha challenge. However, the rest have no login requirement that I can see. They are all .cs files though. Is that why they have to pass the test first?

Feel free to change the title to something more fitting the plan to add logic behind the plus button for cookie/captcha detection 👍

[garfield69]

yes, the indexers written in c# use what ever custom method they choose to implement and the only reason the are written in c# in the first place is because there is something unique about the web site that the standard yaml code cannot support them.
mostly, c# are used for private sites (and so require a login), or any site that requires fetching results from an API rather than web scraping, or have multiple torrents per title, which the yaml cardigann processor cannot handle.
so c# all get tested before going on the active list.

[ngosang]

@ilike2burnthing I did some changes in the past. Could you test again?

[ilike2burnthing]

0.16.118.0 Windows Sandbox - simulating test failure/being unable to reach site, by using a fake proxy.

Same behaviour for all sites listed in OP - if test on clicking Add fails, they aren't added. All new public indexers since then can be added fine. Tested a handful of public indexers that were fine before, and they still get added without issue.

I didn't think that was going to change though, I thought it was remaining open for @garfield69 's comment,

We may need to add logic behind the plus button if possible to detect if the indexer is using the cookie method and invoke the wrench action instead of attempting to add the indexer to the active page.

If that's what you were asking about, then no, clicking Add for the above indexers closes the Add indexer and after a second or two an error displays. The configuration panel doesn't open.

Feel free to nab me for any other testing.

[ngosang]

@garfield69 I have been investigating and this problem can't be fixed in the UI, needs changes in the C# too. Meanwhile you can change public trackers with cookie to => semi-private (with 1 note in the yml or something).