Jackett: [KaraGarga] Multiple Enhancements

[RoloSoze]

• [x] 1. Make Jackett search 'Title' instead of 'Torrents'

On KaraGraga's website, the user has the option to search Torrents, Title, Director/Artist, Uploader, Year (or range), or IMDb ID, with the default option being 'Torrents', which is what Jackett uses. The problem with this option is that any given search returns many partially or completely irrelevant results, since it looks for the keywords everywhere. This essentially renders the search useless in Jackett, unless the search filter is used.

I assume most Jackett users search for a movie using its title, which is why I believe that Jackett should search using the 'Title' option instead of the 'Torrents' option.

• [x] 2. Auto-add quotation marks to the search string AKA ANDMATCH

Even when 'Title' is used, the tracker still returns a lot of irrelevant results. One way to solve this would be to have Jackett auto-add quotation marks to every search string (i.e. andmatch), similar to what was done for KAT in #1049. This returns accurate search results.

E.g. Try searching for Last Year at Marienbad on KG's website, either under Torrents or Title - you'll get countless results.

Now search for "Last Year at Marienbad" (with quotation marks) on KG's website using Title - you'll get 6 results, which are all the results available for this movie (when searched using the English title).

Another example: Search The Spy Who Came in from the Cold, many irrelevant results.
Try searching, "The Spy Who Came in from the Cold" with quotation marks and under 'Title', you'll get 4 correct results.

• [x] 3. Ability to search using 'Title Year' + Returning the full release name/filename in Jackett

March 2020 edit: KG does not include the year in the movie title, but displays the year in a separate column on the browse/search page. I think it's ideal if the year can somehow be shown in Jackett's search results, and the user should be able to include the year in the search query like with any other tracker.

Also, whenever possible/available, it would be ideal if the full release details/filename (i.e. title, year, resolution, source, video codec, audio codec, and release group) are shown in Jackett.

• [x] 4. Correct the 'Published' date in Jackett

KG includes the 'Added date' for all its uploads, yet Jackett shows incorrect Published date and in inconsistent formats - some are in days, some in minutes. Can this be corrected?

• [x] 5. Add the ability to search via IMDb ID & show IMDb URL - [March 2020 edit]

As mentioned under point number 1, IMDb ID can be used to search KG, but this is currently not configured in Jackett #4859. Moreover, the IMDb link is shown next to each release, so it should also appear in Jackett

• [x] 6. Double-check the completeness of the category mapping, accuracy of download discounts, and whether dead torrents are shown in Jackett - [March 2020 edit]

I'm not entirely sure whether all categories are correctly mapped, or whether download discounts, including freeleech, are being correctly reflected in Jackett. The same goes for dead torrents, which should be shown in Jackett for all categories. KG seems to be completely broken in Jackett at the moment.

• [x] 7. Titles with non-English letters: Not displayed correctly, no results when searching using the non-English title - [March 2020 edit]

Try searching Jackett for Marienbad. Some results, which have French letters, are not displayed correctly in Jackett L'ann�e derni�re �. This needs to be fixed.

We should also ensure that when searching Jackett using non-English letters (e.g. L'année dernière à Marienbad), correct results should be returned. Right now, this returns nothing. See also #7673.

Thank you!

Want to back this issue?

Post a bounty on it! We accept bounties via Bountysource.

[adamwinn]

I can take a look at this

[adamwinn]

@RoloSoze can you give me an example in 3. like you did in 2.?

[adamwinn]

@RoloSoze I've tried to fix number 4 already but I can't figure it out. KG displays their dates in a non-standard format. Someone else can fix it maybe.

[adamwinn]

1) and 2) fixed with #1249

[RoloSoze]

@adamwinn Thank you! I think it's best to see how the fixes to 1 & 2 work out before attempting 3. It might not even be needed. 3 basically applies to any movie search that has a year in it. With most trackers, I always include the movie's year in my search string since this usually returns more accurate results, but KG does not include the year in the movie's title.

Let me test the fixes to 1 & 2 when the new version is released, and then I'll discuss 3 with you in more detail, if required.

As for 4, thank you for trying. Let's see if someone is able to fix it. If not, that's fine, it's not a deal breaker :)

[lmsantino]

Do they have a new system for their downloading?

I get the following error when trying to download a torrent from Jackett:

05-30 21:59:25 Error CardigannIndexer (karagarga): Download selector a[href^="download.php?id="] didn't match:
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL /download.php was not found on this server.</p>
</body></html>

05-30 21:59:25 Error Error downloading karagarga https://karagarga.in/download.php?id=129225 System.Exception: Download selector a[href^="download.php?id="] didn't match
  at Jackett.Indexers.CardigannIndexer+<Download>c__async7.MoveNext () [0x0032e] in <31d48d5ba5e14d74937fe73ef8cba7c4>:0
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <dbb16e0bacdc4a0f87478e401bc29b6c>:0
  at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0004e] in <dbb16e0bacdc4a0f87478e401bc29b6c>:0
  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x0002e] in <dbb16e0bacdc4a0f87478e401bc29b6c>:0
  at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x0000b] in <dbb16e0bacdc4a0f87478e401bc29b6c>:0
  at System.Runtime.CompilerServices.TaskAwaiter`1[TResult].GetResult () [0x00000] in <dbb16e0bacdc4a0f87478e401bc29b6c>:0
  at Jackett.Controllers.DownloadController+<Download>c__async0.MoveNext () [0x00168] in <31d48d5ba5e14d74937fe73ef8cba7c4>:0

Anyone have any ideas? An example download file is: https://karagarga.in/down.php/51522/Hrspiel%20-%20Mauricio%20Kagel%20-%20Die%20Umkehrung%20Amerikas.mp3.torrent?passkey=&authkey=

Not really sure how to go about editing the .yml file myself.

[kaso17]

@adamwinn can you have a look at this download issue too?
Looks similar to the waffles issue.
Having to use the top level download block for indirect downloads and a replace filter to generate the download link looks odd to me (But I don't have access)

[lmsantino]

Has anyone figured this out for waffles yet?

[adamwinn]

I wrote @kaso17 an email for clarification but haven't heard back yet.

[lmsantino]

@kaso17 It appears that you've fixed this for waffles.ch; could you look at karagarga as well?

[kaso17]

@0naicuL please try the changes from 1d0790471fc65dddd3a7122375339db8d285f3f8 (I can't test it as I've no access).

[RoloSoze]

Still an issue. Can anyone fix this? @kaso17 @garfield69

[garfield69]

If you send an invite to garfieldsixtynine @ gmail.com I can take a look.

[RoloSoze]

@garfield69 I'll keep you posted on this.

[wkeppler]

@rolosoze I've been working on a fix but I haven't gotten very far yet. It seems like you'll have to get in touch with a mod to get kaso/garfield an invite.

[RoloSoze]

@wkeppler They'll get back to me on this.

[RoloSoze]

@garfield69 Check your Gmail.

[grocovid]

I managed to fix karagarga torrent download issue. Just replace in karagarga.yml

download:
selector: a[href^="details.php?id="]
attribute: href
filters:
- name: replace
args: ["details.php?id=", "download.php?id="]

with
download:
selector: a[href^= "/down.php"]
attribute: href

skipping the replace part. It works. I'm still trying to figure out the rest of the yml syntax to try and make the imdbid search work.

[grocovid]

And for the imdb part, I simply changed things to
modes:
search: [q]
movie-search: [q, imdbid]
and
inputs:
search: "{{ if .Query.IMDBID }}{{ .Query.IMDBID }}{{else}}{{ .Keywords }}{{end}}"
search_type: "{{ if .Query.IMDBID }}imdb{{else}}title{{end}}"

See https://pastebin.com/bhCzAkRG.
My apologies for probably ignoring github rules - these are my first posts here

[RoloSoze]

@cadatoiva Since you're a PTP member, I might be able to help you with a Karagarga membership if you're willing to work on these issues. Please get in touch with me via email.

[cadatoiva]

Still waiting on that invite, @RoloSoze

[RoloSoze]

Still waiting on that invite, @RoloSoze

@cadatoiva I sent you an email yesterday with some updates. Waiting for your response.

[cadatoiva]

@RoloSoze I found a bunch of your emails in my spam filter. Not sure how they got there, I'm replying to them all now.

[RoloSoze]

@RoloSoze I found a bunch of your emails in my spam filter. Not sure how they got there, I'm replying to them all now.

Ah! No wonder I haven't been hearing back from you :) And your responses just ended up in my Junk Email, too. Please mark me as a Safe Sender or save me in your Email contacts to avoid this issue in the future! I just added you to mine. Thank you.

[RoloSoze]

@cadatoiva Some of these issues may have been addressed, partially or completely, since my initial post. I'll let you be the judge on whether the solution(s) were done correctly from an implementation standpoint.

[RoloSoze]

Related #4419

[RoloSoze]

@cadatoiva I just want to make sure that my latest email to you about this, sent 2 days ago, didn't go to Spam. Thanks.

[cadatoiva]

Didn't go to spam, just haven't opened my email in about a week. Replying right now.

[RoloSoze]

Didn't go to spam, just haven't opened my email in about a week. Replying right now.

@cadatoiva Gotcha. I just replied to your email. Please check.

[Norin-Radd]

👍

[RoloSoze]

@Norin-Radd Do you have a spare invite by any chance?

[Norin-Radd]

let me see, just a moment.

[Norin-Radd]

sorry my user class doesn´t let me invite people yet. I am not a Power User yet.

[garfield69]

In the first post, option 2 and 3 clash.
You cannot encapsulate the query in quotes (to promote an exact match search) and include the year at the same time, because the year is not part of the title as far as the web site is concerned.
So a search using "The Spy Who Came in from the Cold 1965" will fail.

Also, using exact matching will not allow for a successful search for spy came cold, where as doing a loose search and using jackett's andmatch we can fetch results that contain all the keywords.
Since the web site orders results by default for most matches first, this should yield best results.

So I will implement this first, and revisit exact matching if andmatch turns out unsatisfactory later.

[RoloSoze]

In the first post, option 2 and 3 clash.

You're right. I kept updating the issues over a long period of time, so I didn't notice that these two clash.

whereas doing a loose search and using jackett's andmatch we can fetch results that contain all the keywords.

Sounds like this is the way to go then. Go ahead.

Also, can the year be fetched from its column and displayed next to the movie title as if it were a part of it?

[garfield69]

yes.

[garfield69]

to anyone familiar with the web site:
so freeleech and featured are clearly download cost 0
and all uploads past 100% get a default 10% upload bonus
MoM which are CURRENT get a 50% upload bonus, presumably on top of the 10% or not?
The indexer currently shows 60% upload bonus for MoM CURRENT.

is there more?
for instance, the indexer seems to be set up to show that HD uploads will return 30% bonus
and HD CURRENT 80%, but I cannot see any supporting info for this, is it valid or obsolete?
Any other download upload costs features?

[RoloSoze]

to anyone familiar with the web site:
so freeleech and featured are clearly download cost 0
and all uploads past 100% get a default 10% upload bonus
MoM which are CURRENT get a 50% upload bonus, presumably on top of the 10% or not?
The indexer currently shows 60% upload bonus for MoM CURRENT.

is there more?
for instance, the indexer seems to be set up to show that HD uploads will return 30% bonus
and HD CURRENT 80%, but I cannot see any supporting info for this, is it valid or obsolete?
Any other download upload costs features?

@adamwinn @lmsantino @wkeppler @grocovid @Norin-Radd @drewbitt @dmucli @savahu @deathblankets @fim @duramato @xpirat @awinnpii

You guys are all Karagarga members. @garfield69 is currently working on fixing its Jackett integration and functionality. If any of you is familiar with KG's upload and download bonuses and discount rules, please help us by answering @garfield69's question.

Additionally, if there are any Karagarga issues that you know of - that are not already listed in the OP of this issue - please feel free to detail them here.

Thank you.

[RoloSoze]

@garfield69 If none of the above 13 users responds, just go with your interpretation of the instructions on the FAQ page. Also, regarding the HD uploads, see what it says on the description page of some of the HD torrents, both old and 'current' ones. I can double check your logic/understanding with the staff at a later stage.

[Norin-Radd]

I still don´t have an invite.

[Norin-Radd]

[RoloSoze]

@Norin-Radd Thank you. However, the question was not about providing an invite. It was about confirming @garfield69's understanding of the upload & download bonuses and discounts as mentioned in https://github.com/Jackett/Jackett/issues/1241#issuecomment-636278039

It would be great if you're able to clarify that.

[Norin-Radd]

i only see this

[Thank you!]

[garfield69]

Jackett 0.16.585

[duramato]

Just gave it a test and it doesn't seem to work.
Karagarga seems to be requiring a captcha on login now

[RoloSoze]

@garfield69 The work of a superhero! Thanks a lot, boss!

@duramato I didn't need to solve a CAPTCHA myself, but since there is one, @garfield69 please feel free to implement that.

Also, I'm assuming that fetching any details from the torrent's details page requires a separate request? Because the full filename/release name, when available, is found on the details page under 'Rip Specs'.

Example:
The following release appears like so in Jackett, which is how it is shown on the browse/search page on KG's website:
Dan duzi od godine AKA A Day Longer Than a Year 1971 Yugoslavia

On the torrent's details page, the actual filename is shown, which does not appear in Jackett:
Dan.duzi.od.godine.1971.1080p.WEB-DL.AAC2.0.H.264-KG

Whether or not the filename/release name is actually complete/informative depends of course on whether the file itself was named properly.

If displaying the latter is somehow possible, it might be worth it as long as we're not sending two requests to KG.

[garfield69]

@duramato I don't get a captcha during login, so you are going to have to provide a full enhanced log file should include the html you are presented, and give me something to work with.
see https://github.com/Jackett/Jackett/wiki/Troubleshooting#how-to-provide-a-jackett-enhanced-log-for-a-ticket

[garfield69]

@RoloSoze a href="/down.php/257531/Dan.duzi.od.godine.1971.1080p.WEB-DL.AAC2.0.H.264-KG.mkv.torrent?passkey=**redacted**" is available in the results
are you saying that you would prefer the title to be
Dan.duzi.od.godine.1971.1080p.WEB-DL.AAC2.0.H.264-KG.mkv
rather than
Dan duzi od godine AKA A Day Longer Than a Year 1971 Yugoslavia
?

[RoloSoze]

Are you saying that you would prefer the title to be
Dan.duzi.od.godine.1971.1080p.WEB-DL.AAC2.0.H.264-KG.mkv
rather than
Dan duzi od godine AKA A Day Longer Than a Year 1971 Yugoslavia
?

@garfield69 I wouldn't say I prefer the first over the second. The reason is the first is the actual filename of the release. It will only be useful if the file itself has an informative name, like the example above. By informative, I mean it includes details such as title, year, resolution, source, video codec, audio codec, and release group. Scene and P2P releases that have not been renamed will have all of these, and ideally, every release should have those, but in reality, that's not the case in KG. Some will have a name such as Movie title.mp4, so returning this instead of what we're returning now would be a bad idea.

What I propose instead is returning both the above titles in a manner similar to how PassThePopcorn's API returns results - see https://github.com/Jackett/Jackett/issues/6783#issuecomment-599278807 - Start reading from where it says, "The best implementation example I've seen in Jackett."

Jackett users should see:

Dan duzi od godine AKA A Day Longer Than a Year 1971 Yugoslavia
[Dan.duzi.od.godine.1971.1080p.WEB-DL.AAC2.0.H.264-KG.mkv] (IMDb logo & upload/download info go here)

So that if the filename is actually informative, the user can see the info in Jackett without having to open the torrent's details page on KG's website. Whether you want to have the second title start in its own line is up to you, but I think it's aesthetically better. I was actually going to ask cadatovia to move the tags between the square brackets to the second line in PTP's results instead of letting it overflow. I think it would look better.

I do not think many users would have an issue with both titles being displayed, but just in case - and if possible - we can add a checkbox on KG's configuration page to make displaying the filename optional?

[dmucli]

Thanks for all the work @garfield69!

I'm seeing this error now when trying to test the tracker:

Error | System.Exception: Found no results while trying to browse this tracker at Jackett.Common.Services.IndexerManagerService.TestIndexer(String name) in /Users/runner/runners/2.169.1/work/1/s/src/Jackett.Common/Services/IndexerManagerService.cs:line 263 at Jackett.Server.Controllers.IndexerApiController.Test() in /Users/runner/runners/2.169.1/work/1/s/src/Jackett.Server/Controllers/IndexerApiController.cs:line 130 at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.TaskOfIActionResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.g__Awaited\|12_0(ControllerActionInvoker invoker, ValueTask`1 actionResultValueTask) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.g__Awaited\|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.g__Awaited\|13_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited\|19_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted) at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited\|17_0(ResourceInvoker invoker, Task task, IDisposable scope) at Microsoft.AspNetCore.Routing.EndpointMiddleware.g__AwaitRequestTask\|6_0(Endpoint endpoint, Task requestTask, ILogger logger) at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context) at Jackett.Server.Middleware.CustomExceptionHandler.Invoke(HttpContext httpContext) in /Users/runner/runners/2.169.1/work/1/s/src/Jackett.Server/Middleware/CustomExceptionHandler.cs:line 70
-- | --

[garfield69]

@dmucli The indexer is working me which suggest the problem is at your end.
First try removing your active indexer using the Jackett dashboard trashbin icon, and going to the add-indexer page and adding it afresh.
If this does not work, then please provide a full Enhanced log file which should include the HTML returned by the site so I have something to work with, as the stack trace you provided is not useful in this case.
see https://github.com/Jackett/Jackett/wiki/Troubleshooting#how-to-provide-a-jackett-enhanced-log-for-a-ticket

[RoloSoze]

@duramato @dmucli Guys, @garfield69 does not have permanent access to Karagarga. Therefore, the sooner you can provide the enhanced logs he requested for your respective issues, the more likely he is to be able to resolve them. Your timely cooperation is highly appreciated. Thank you.

[garfield69]

@RoloSoze I can easily add [filename] to the title, but I cannot insert a linebreak into the title body to force a two line presentation. The Title string has to conform to Torznab specifications, which does not support embedding HTML tags to prettify the string.

[RoloSoze]

@garfield69 No problem. No need for the line break then. Thank you.

[dmucli]

Please disregard the error mentioned above as I was able to fix it by relogin.

[RoloSoze]

@garfield69 Check your email. I may have found the CAPTCHA error message.

[RoloSoze]

@garfield69 Phenomenal work, @garfield69! Thank you so much for taking care of this 3+ year old issue!

[dmucli]

Thank you - working great now!

[Norin-Radd]

seems its working radarr grabbed a movie from karagarga without I noticing eheheh

thanks!

[garfield69]

Jackett 0.16.598 for login with captcha