Sonarr: Stalled torrent detection

[ghost]

Hi all
I understand that there is a rule that the torrent have to have atleast 1 seed to fetch the torrent.

But often i get that is no seeds at all when i look it up in Transmission.

So i suggest that we can have a setting where we choose how many seeds we want.

And since that is not possible to see on all trackers we should also have a rule that if nothings happens in like 15-30 minutes the torrent are deleted and blacklisted and a new one will be downloaded.

Thanks in advance and thanks for a great application.

[kayone]

I think there are two separate issues here,

1. add support to configure minimum number of seeds, very reasonable, the question is what should be the scope? I want to say global setting. but maybe there are reasons to do it per tracker?
2. torrent failure cases. when should it be considered failed? slow speed? over what period, etc. this one is more complicated both from requirement perspective and implementation.

[ghost]

I would say global is ok per tracker is overkill I guess.

The 2:nd one I understand are the most complex one but that would also give the most to the users.
I would say just a setting where I set X minutes and if the torrent is not done delete and block it cus if I set it on 60 minutes I won't get it at all if I wait longer. I don't need the episode that second so I can wait for a while before it terminates the torrent.

That's my thoughts on the matter I not to picky I just want some functionality that brings me close to no manual work at all :)
And I would say that torrent is the only thing that makes me do manual work right now :)

[markus101]

I would say just a setting where I set X minutes and if the torrent is not done delete and block it cus if I set it on 60 minutes I won't get it at all if I wait longer.

That might work for a seedbox with very fast download speeds, but a user shouldn't have to guess how long it will take for a download complete, what if its a 2160p release of a full season vs a 480p single episode, those wouldn't have the same download time.

One consideration is the internet being down, if Sonarr and the client and communicate, but the client can't connect to the internet Sonarr can't start blacklisting every incomplete torrent.

There will almost always be some level of manual work required with Sonarr, especially when it comes to complex issues such as this.

[ghost]

Yeah I see the problems just wishing we had something to help us make it more automatic
:)

[tristaoeast]

Sorry for "re-opening" this thread, but I believe that at least the solution proposed for problem 1 would be a very reasonable way of avoiding stalled torrents, and like @spyvingen said, I also believe that per tracker might be a bit of an overkill, but that also depends on how it would impact the overall performance of Sonarr and how costly one and the other solution would be to implement.

Regarding issue 2, I believe that a better workaround would be the possibility to select how Automatic Search prioritizes the results it gets from the trackers. I know someone suggested this in another thread, but what they were suggesting was to make it possible to configure it in a fine-grained manner, which could be too complex and produce undesired results. Instead I would suggest creating three or four different priority "profiles" fo the torrents, similarly to what you have done with the qualities, from which the user could select the one better suiting its needs. Right now I'ḿ having a problem with older seasons, where, within the same quality (an episode of Marvel's Agent Carter from Season 1 at 720p is an example I had), the torrent with the lowest number of seeds (just one seed which eventually disappears and the download stalls) is getting selected, where there are other torrentz with the same quality, equally acceptable by the profile and a lot more peers that are not being downloaded. So a profile suggestion would be, for example, to sort by number of seeds/peers within the same quality, which isn't happening from what I'm seeing when I use Manual Search and select to order the results according to Auto-Search criteria.

Sorry for the long post, and thanks in advance for your time. Please let me know what do you think ;)

[markus101]

@tristaoeast thats really more about better prioritization when choosing results, not detecting stalled torrents and is something we're working on.

[tristaoeast]

@markus101 I know it's not really about detecting stalled torrents, I was thinking of it more like a workaround. But it's great to hear that you are working on improving prioritization :) Can I help with that in any way? Is there a discussion anywhere on how you are planning to improve it? :)

Keep up the execllent work :D

[ntcong]

Is there any updates on the improvement? I think a global configured minimum seed required is enough, or per indexer. Per tracker is overkill imho.

[rux616]

Something needs to be done about this issue. I am finding that Sonarr is not grabbing episodes because the RSS feed (eztv.ag in this case) has an issue and populates the seeds field with 0's.

At the very least, there should be an option to turn this feature off with a warning triangle next to it saying something about possibly causing stalled downloads, etc. Additionally, items with 0 seeds should still be last in priority, that is, used only if nothing else is available.

[kiwixz]

Yes we really need an option to ignore seeders count !

[limetime]

Hi just wanted to add some support to this thread.

My thoughts on improving this would be to sort of work with Sonarr's way of working rather than to create more on top of it. By this I mean:

1. Through my profile settings I can already choose both Torrent and Usenet, I add a show and search.
2. If a torrent gets picked up but it's really slow then let me have a parameter that globally forces another search after certain amount of elapsed time. This would then obey the same rules as normal, in my case preferring nzb and then torrents as a backup (with more seeds than the currently downloading file). If there isn't anything new to pickup then carry on, if there is something better out there, give it a go.

This would be a good way of letting me sure that the files I'm downloading after the best available rather than having to manually search over and over again when I notice that a torrent has stalled.

Hope that makes sense and thanks for listening :)

[bengalih]

If this is still being considered I would vote for speed check or some type of calculation based on time remaining of the torrent vs download size. This would probably have to be salted with a bandwidth number provided by the user (or computed via an under the hood speed test).

So an 8GB movie on a 100mbit connection should take what...10-12 minutes assuming all b/w available?
So a tolerance value would need to be provided by the user as to how long they are willing to wait. Not sure how to best indicate this value e.g.: it would start at 100% (since it is silly to expect it to take quicker than my b/w allows). So if I specify 1000% (or maybe just call it a Tolerance of 10) I would be willing to wait 100-120 minutes for that torrent to download.

It could get quite a bit more complicated with controlling this, and I think there may need to be different profiles as well. For instance most users I expect simply want their new episodes as quickly as possible whereas others who want to archive may not care that it takes 2 weeks to download an entire season in 1080p+.

There is a can of worms to open here because there might be network issues/congestion due to many things so this might have to be measured over a range of time. I have two other suggestions that offer a similar functionality through a different approach.

1) Set a global tolerance for downloads of each quality. This can be based on the already existing profile sizes and a bandwidth the user provides (this allows tweaking from both ends then). If a download isn't processed within this time (or this time plus a configurable multiplier), start a new download of the next option. Whichever one completes first, cancel/delete the other one.

2) Add a better visual indicator to the Activity queue based on most of the data already there. Some sort of exclamation point that clearly says "Hey this thing is taking too long for what it should" and allow a one-click option to either cancel/blacklist and go with the next in line or to continue on with the behavior I mention in #1 (start a second and see who wins).

[mccorkled]

Any news on the implementation of this?

[D4rkSl4ve]

This would be an awesome way to less maintenance. For instance, I am having to look into my Deluge every several days as there are many torrents just sitting there at 0kb or just stalled/non-finished torrents. So, I have to login to the @rr software, go to Activity/Queue, click the 'X' and it removes it from Deluge, and quickly searches for a new torrent. If there was a global where I can say: if Activity/Queue is at 0kb for longer than 'X' time, remove, blacklist, and search again. Or a torrent at 0 speed download for "X' time, remove, blacklist, and search next torrent. A suggestion that would minimize the maintenance on having to check the downloader; which ever it is, as most used have API that can supply such info back, just as the @rr softwares read them to update the Activity/Queue screen.

Pretty please...

[CoderKiwi]

At a bare minimum, it may be nice to have a sort of timeout feature. So if the torrent isn't downloaded in say a week or a few days of it being added then remove it and blacklist it and search for another.

That circumvents the "what if Sonarr isn't connected to the internet and it starts blacklisting everything" issue. And it solves the edge case of torrents that are just never completed.

[untoreh]

using sonarr api it is pretty easy, not much logic here, maybe a strikes count >3 to push the deletion.
Timeout seems kind of meh to me...if a torrent doesn't have seeds what are the chances of it getting reseeded? you can wait forever...

#!/bin/bash

apikey=4f3a98b538594bd8a58f162fb4cce2b5
host=localhost:8989
sleep=600
blacklist=true

while :; do
    queue=$(curl -s -X GET "$host/api/queue?apikey=$apikey" | jq -r '.[] | "\(.id) \(.status)"')
    warning=$(grep "Warning" <<< "$queue")

    IFS=$'\n'
    for t in $warning; do
        id=${t/ *}
        curl -s -X DELETE "$host/api/queue/$id?apikey=$apikey&blacklist=$blacklist"
    done
    sleep $sleep
done

[rcdailey]

How many more years must we wait?

[D4rkSl4ve]

How many more years must we wait?

@markus101 v3 alpha has this?

[markus101]

No, if it did this issue would be closed and in the v3 milestone.

[springjools]

Difficult to make decisions? Seems like all suggestions and proposals would be better than the status quo. What do you need? A pr?

[leoshusar]

I started using Sonarr about a week ago and I'd love this feature too, something like "if speed lower than X for Y time, then mark as failed". Main problem for me is that some websites lie about seeders so even if I have set minimum 5 seeders, it finds torrents with 1 seeder and speed is 5 kb/s, sometimes it stops completely

[Yoshi2889]

Perhaps a check where, if after X time period, it hasn't at least downloaded Y MB/KB/..., it would drop the torrent?

I'd suppose the defaults could be pretty high up, I think a rule of 5 MB in 2 days would catch most stalled torrents and cause relatively few false positives. Just 1 day could potentially be dangerous when the torrent is only being seeded at a specific time, e.g. when it's only being seeded at night.

[wvanbreukelen]

An torrent timeout feature is definitely needed in my opinion as well. As for now, I will try if I can arrange something similar using a cronjob and the API. I will try to report back.

[springjools]

Can't you just use the autoremove plug in and check for availability?

On Mon 2 Sep 2019, 14:33 Wiebe van Breukelen, [e-mail]
wrote:

A torrent timeout feature is definitely needed in my opinion as well. For
now, I will try if I can arrange something similar using a cronjob and the
API. I will try to report back.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/Sonarr/Sonarr/issues/958?email_source=notifications&email_token=ABNDLLRG3QA4RBN6XYYI6NDQHT2Y7A5CNFSM4BVBL6A2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5VSAEI#issuecomment-527114257,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABNDLLV3V664TLNMLRO2KOLQHT2Y7ANCNFSM4BVBL6AQ
.

[wvanbreukelen]

Can't you just use the autoremove plug in and check for availability?
…
On Mon 2 Sep 2019, 14:33 Wiebe van Breukelen, @.*> wrote: A torrent timeout feature is definitely needed in my opinion as well. For now, I will try if I can arrange something similar using a cronjob and the API. I will try to report back. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#958?email_source=notifications&email_token=ABNDLLRG3QA4RBN6XYYI6NDQHT2Y7A5CNFSM4BVBL6A2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5VSAEI#issuecomment-527114257>, or mute the thread https://github.com/notifications/unsubscribe-auth/ABNDLLV3V664TLNMLRO2KOLQHT2Y7ANCNFSM4BVBL6AQ .

Good suggestion. Unfortunately, at the moment I am using a Docker container with Transmission to prevent VPN leaks (it seems that the autoremove plugin is only available for Deluge).
However, I am curious if it might be possible to go with a more universal approach with the API, so that all radarr/sonarr supported torrent clients can be used. Otherwise, if I have time, will try to code it myself and create a PR.

[SiemdeNijs]

I was wondering if the best approach could be to somehow specify download client states (retrieve in plain text). Then for example on 'error', or 'stalled' select an action.

Such as: (the same action as when you manually remove a download under activity, and have it set up) it can blacklist the release and search again, or a action to recheck the torrent, or anything such as. Same would be handy for 'Error' state downloads.

Sonarr seems to already give the same descriptional errors as the download client, on the activity page, so these seem retrievable. And since the function of blacklisting and searching again is kinda already present, perhaps it would be possible ?

[Treverr]

@wvanbreukelen, did you ever get anything successfully set up?

[roberestarkk]

I honestly don't see why Sonarr can't meet the download client somewhere in the middle.

In an ideal situation (and I believe this is much like how Usenet failure handling functions right now), the download client would internally decide whether a download had failed, and then do something to indicate that in some way to whoever might want to know. Be it via torrent removal, or a special tag, or whatever anyone else can come up with.
Then whoever was interested in the state of torrents (which Sonarr and Lidarr and whatnot most definitely are), would be able to observe that indicator, and react to it.

All of this is entirely possible with the existing features of Sonarr. It can already tell what level of progress a torrent is at, so it can read the state of the torrent. It can also read the state of the tags as it actively uses tagging to determine whether it is in 'control' of a torrent.
I don't believe it is unreasonable to assume it could also detect the removal of the torrent from the client (thus being unable to retrieve the progress even though it has been accepted by the download client), or the addition of a tag, to communicate that it had failed.

All the onus of determining whether the download had 'failed' would still be on the download client (which is a fair point markus mentioned in #3444 ), all Sonarr would need to do is decide that the download client making a certain change to the state of the download is a valid _indication_ of a download failure and handle it.
We're basically just asking for a communication protocol to be defined for Torrent failure that uses a feature which already exists in the supported torrent clients and seems entirely within the existing capabilities of Sonarr, rather than requiring all download clients to implement API calls to Sonarr (and Radarr, and Lidarr, etc).
Just as Sonarr shouldn't be responsible for reaching into the guts of the download client to determine download failure, the download client shouldn't be responsible for reaching into the guts of Sonarr to tell it the download had failed.

[springjools]

Well, the download can fail for many reasons: it could be phony contents (which should be blacklisted), or it could be stuck because people hit and run. Anyway, if you use deluge, you can use https://github.com/springjools/deluge-autoremoveplus/releases to pause all torrents based on those conditions (in effect your indication of a download failure), and then manually remove all paused downloads, or let the plugin do so automatically.

It works for sonarr v3 but not yet for radarr or lidarr, because they can not yet blacklist without remove from client. You can use it with lidarr and radarr through the pause indication work around though.

What I use for stalled detection is X time since last transfer or X time since complete seen.

[roberestarkk]

I actually raised an issue on your project because I can't use it :(
The upshot of which is I still can't use it because I don't have enough control over the docker container it runs in to upgrade the version of Python it uses or provide it extra packages.

At the moment I am using the Python 2 version of the autoremove plugin, and just manually checking and flagging downloads as failed in Sonarr when I remember so it can try again.
I just don't want to have to do that manually, as it defeats the purpose of using Sonarr to a degree

[springjools]

@roberestarkk Can you contact the owner of your docker container and ask him/her to install the requests module? (pip install python-requests)

[roberestarkk]

I can, and have tried, and they refused. It's very upsetting.

[springjools]

Well, it's impossible to communicate with sonarr api without the requests module (or something similar)

I guess I could produce a version of the plugin without that module that only pauses the torrent, and then you can remove all paused torrents from sonarr periodically. Paused torrents still don't take up slots or anything like that.

[roberestarkk]

Yeah, no you're fine. The old python2 version works fine to remove them, I just have to remember to compare with Sonarr and mark them as failed.
I really just need a new hosting provider though, I don't even think they'd update Sonarr even if this feature was added.

I still stand by the whole 'Sonarr should be able to respond to an indication of download failure from a torrent client" stance though.

[y0da822]

I can, and have tried, and they refused. It's very upsetting.

Can’t you exec into the container and manually install the requests module? I was able to using Linuxserverio deluge. Maybe not but just curious.

[roberestarkk]

Guys I do appreciate all the tech support, but I'm in a managed shared environment because I'm cheap, and they use docker to provide plug-n-play addons, so they have docker locked down and unavailable to us mere peasants.
I've sacrificed control for cheapness, so there's no hope for me until I migrate elsewhere.

There is hope for Torrent Download Failure Handling in Sonarr though.
It's just a matter of we as a community deciding on a pesudo-standard and requesting that it be implemented.

To re-iterate what I believe was my strongest point.
"Just as Sonarr shouldn't be responsible for reaching into the guts of the download client to determine download failure, the download client shouldn't be responsible for reaching into the guts of Sonarr to tell it the download had failed."

There should be a way for the Download client to passively advertise that a download has failed, and there should be a way for Sonarr to detect and respond to that if the user wants it to.
I just learned writing this post that Sonarr does support custom scripting for processing rules and such, but by the looks of it there's no way to use the existing hooks to support this particular thing, which is why we're asking for it to be included as a feature in the main application.

I personally believe that way should be 'The torrent was removed from the download client and Sonarr doesn't have the file it was expecting'.
In my opinion, there is no reason that Sonarr shouldn't treat that scenario as a failed download.
I am open to being presented one though.