Sonarr: Occasional failure to connect to indexer until restart

[lutzky]

Describe the bug

I use a local Jackett instance (separate docker container) for all indexers. Occasionally (once every few days), Sonarr will be unable to connect to Jackett, and report all indexers as unavailable; testing the indexers will time out (after about a minute). Restarting Sonarr (from the Sonarr UI itself, System->Restart) works around the problem, and indexer testing will succeed immediately. When the problem is active, contacting Jackett via curl from within the Sonarr container works normally.

Screenshots
N/A

Logs

Sonarr trace log:

18-11-5 20:06:34.2|Trace|Http|Res: 506 [GET] /api/health: 200.OK (2 ms)                                                            
18-11-5 20:06:34.2|Debug|Api|[GET] /api/health: 200.OK (2 ms)                                                                      
18-11-5 20:06:37.3|Trace|Http|Req: 507 [POST] /api/indexer/test                                                                    
18-11-5 20:06:37.4|Debug|Torznab|Downloading Feed http://wamc_jackett_1:9117/api/v2.0/indexers/1337x/results/torznab/api?t=tvsearch
&cat=5030,5040&extended=1&apikey=(removed)&offset=0&limit=100                                                                      
18-11-5 20:06:37.4|Trace|HttpClient|Req: [GET] http://wamc_jackett_1:9117/api/v2.0/indexers/1337x/results/torznab/api?t=tvsearch&c$
t=5030,5040&extended=1&apikey=(removed)&offset=0&limit=100                                                                         
18-11-5 20:06:37.4|Trace|ConfigService|Using default config value for 'proxyenabled' defaultValue:'False'                          
18-11-5 20:07:03.7|Trace|Scheduler|Pending Tasks: 0                                                                                
18-11-5 20:07:19.5|Warn|Torznab|Unable to connect to indexer                                                                       

[v2.0.0.5252] System.Net.WebException: The operation has timed out.: 'http://wamc_jackett_1:9117/api/v2.0/indexers/1337x/results/t$
rznab/api?t=tvsearch&cat=5030,5040&extended=1&apikey=(removed)&offset=0&limit=100' ---> System.Net.WebException: The operation has 
timed out.                                                                                                                         
  at System.Net.HttpWebRequest.RunWithTimeoutWorker[T] (System.Threading.Tasks.Task`1[TResult] workerTask, System.Int32 timeout, S$
stem.Action abort, System.Func`1[TResult] aborted, System.Threading.CancellationTokenSource cts) [0x000f8] in <9b672a45b19f4d52b5f$
8f32c0c91d97>:0                                                                                                                    
  at System.Net.HttpWebRequest.GetResponse () [0x00016] in <9b672a45b19f4d52b5f28f32c0c91d97>:0                                    
  at NzbDrone.Common.Http.Dispatchers.ManagedHttpDispatcher.GetResponse (NzbDrone.Common.Http.HttpRequest request, System.Net.Cook$
eContainer cookies) [0x000f6] in C:\BuildAgent\work\5d7581516c0ee5b3\src\NzbDrone.Common\Http\Dispatchers\ManagedHttpDispatcher.cs$
64                                                                                                                                 
   --- End of inner exception stack trace ---                                                                                      
  at NzbDrone.Common.Http.Dispatchers.ManagedHttpDispatcher.GetResponse (NzbDrone.Common.Http.HttpRequest request, System.Net.Cook$
eContainer cookies) [0x0019e] in C:\BuildAgent\work\5d7581516c0ee5b3\src\NzbDrone.Common\Http\Dispatchers\ManagedHttpDispatcher.cs$
92                                                                                                                                 
  at NzbDrone.Common.Http.Dispatchers.FallbackHttpDispatcher.GetResponse (NzbDrone.Common.Http.HttpRequest request, System.Net.Coo$
ieContainer cookies) [0x000b5] in C:\BuildAgent\work\5d7581516c0ee5b3\src\NzbDrone.Common\Http\Dispatchers\FallbackHttpDispatcher.$
s:53                                                                                                                               
  at NzbDrone.Common.Http.HttpClient.ExecuteRequest (NzbDrone.Common.Http.HttpRequest request, System.Net.CookieContainer cookieCo$
tainer) [0x0007e] in C:\BuildAgent\work\5d7581516c0ee5b3\src\NzbDrone.Common\Http\HttpClient.cs:121                                
  at NzbDrone.Common.Http.HttpClient.Execute (NzbDrone.Common.Http.HttpRequest request) [0x00008] in C:\BuildAgent\work\5d7581516c$
ee5b3\src\NzbDrone.Common\Http\HttpClient.cs:57                                                                                    
  at NzbDrone.Core.Indexers.HttpIndexerBase`1[TSettings].FetchIndexerResponse (NzbDrone.Core.Indexers.IndexerRequest request) [0x0$
04b] in C:\BuildAgent\work\5d7581516c0ee5b3\src\NzbDrone.Core\Indexers\HttpIndexerBase.cs:321                                      
  at NzbDrone.Core.Indexers.HttpIndexerBase`1[TSettings].FetchPage (NzbDrone.Core.Indexers.IndexerRequest request, NzbDrone.Core.I$
dexers.IParseIndexerResponse parser) [0x00000] in C:\BuildAgent\work\5d7581516c0ee5b3\src\NzbDrone.Core\Indexers\HttpIndexerBase.c$
:298
  at NzbDrone.Core.Indexers.HttpIndexerBase`1[TSettings].TestConnection () [0x0000e] in C:\BuildAgent\work\5d7581516c0ee5b3\src\Nzb
Drone.Core\Indexers\HttpIndexerBase.cs:335

18-11-5 20:07:19.6|Trace|NzbDroneErrorPipeline|Handling Exception
18-11-5 20:07:19.6|Warn|NzbDroneErrorPipeline|Invalid request Validation failed:
 -- Unable to connect to indexer, check the log for more details
18-11-5 20:07:19.6|Trace|Http|Res: 470 [POST] /api/indexer/test: 400.BadRequest (100096 ms)
18-11-5 20:07:19.6|Debug|Api|[POST] /api/indexer/test: 400.BadRequest (100096 ms)

Running curl directly on the URL as shown in the log, from within the Sonarr container:

# time curl 'http://wamc_jackett_1:9117/api/v2.0/indexers/1337x/results/torznab/api?t=tvsearch&cat=5030,5040&extended=1&apikey=redactedapikey&offset=0&limit=100' -s | head

<?xml version="1.0" encoding="UTF-8"?>
<rss version="1.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:torznab="http://torznab.com/schemas/2015/feed">
  <channel>
    <atom:link href="http://wamc_jackett_1:9117/jackett/" rel="self" type="application/rss+xml" />
    <title>1337x</title>
    <description>1337X is a Public torrent site that offers verified torrent downloads</description>
    <link>https://1337x.to/</link>
    <language>en-us</language>
    <category>search</category>
    <image>

real    0m0.538s
user    0m0.008s
sys     0m0.009s

Link to debug or trace log files.

System Information

• Sonarr Version: 2.0.0.5252
• Operating System: Docker 18.06.0-ce on Ubuntu 18.04
• Mono version: 5.16.0.179

UI Bugs:
N/A

Additional context
N/A

[aDarkling]

Same thing happened to me. Sonarr reported that 7 indexers weren't connecting. After I restarted, all of them passed their tests.

[mechiah]

Note this issue here: https://github.com/Sonarr/Sonarr/issues/2777

Your issue might be related to an issue with Rarbg reporting blank RageIDs, and breaking Sonarr. If you are having that issue, disabling RSS sync for your Rarbg indexer in Sonarr may be a suitable workaround for you, until this is fixed.

[lutzky]

@mechiah this is different from #2777, because in that case performing the test manually succeeds. In my case, the manual test times out.

[mechiah]

Manual tests usually, but not always, timed out for me as well, FWIW. Disabling RARBG resolved it for me.

Could be different issues, in the end, but my symptoms reflected both 2777 and 2802 in the limited time I monitored... before disabling Rarbg. I'm up for about 26 hours without indexers being disabled (previous record was about 8 hours).

[lutzky]

I don't have Rarbg enabled, I exclusively use jackett for all of my indexers.

[mechiah]

Sorry for not being specific. Rarbg isn't an integrated indexer in Sonarr (AFAIK?). I was referring to using Rarbg via Jackett.

[lutzky]

It does appear to be integrated (it shows up in the list, at any rate), but I use 1337x, EZTV, KickAssTorrent and ThePirateBay.

[jeffkeller87]

Same issue here. I am using RARBG through Jackett and not via Sonarr integration. Will try to disable RSS sync and see if that resolves the problem in this case as well.

[jeffkeller87]

Problem remains.

[lutzky]

I'm not entirely sure if this is relevant, but: I had this problem again, and noticed that Sonarr was using 100% CPU (1 whole core of a 4-core machine); when I restarted it using the UI, problem persisted, and now it was using 200% CPU (2 cores). I then restarted its whole docker container (docker restart), CPU usage went back to ~0% and the problem went away. I'll update again if I see the problem happening separately from a 100%-CPU-usage situation.

I'm also instating a cron job to restart the container every 12 hours as a workaround.

[Celoxocis]

i just recently came across the exact same issue as described by lutzky. once the sonarr container was restarted. sonarr was able to connect without any issues.
i did however notice that the issue started to happen with an upgrade to the latest available docker:
(running 16.04 LTS)

Client:
Version: 18.09.0
API version: 1.39
Go version: go1.10.4
Git commit: 4d60db4
Built: Wed Nov 7 00:48:57 2018
OS/Arch: linux/amd64
Experimental: false

Server: Docker Engine - Community
Engine:
Version: 18.09.0
API version: 1.39 (minimum version 1.12)
Go version: go1.10.4
Git commit: 4d60db4
Built: Wed Nov 7 00:16:44 2018
OS/Arch: linux/amd64
Experimental: false

[Papsnoek]

I have the exact same problem as the original Post. I have even removed the Rarbg indexer completely to be safe. but no luck.
Running on a Odroid XU4
ARMBIAN 5.60 stable Ubuntu 18.04.1
Docker CE 18.09.0

[lutzky]

The "restart every 12 hours" cronjob workaround is successful albeit hacky.

[markus101]

Mono version: 5.16.0.179

Is everyone else also running mono 5.16

Since this issue is recent it seems likely it's related to a change in mono causing requests to timeout. I personally have no run into this with 5.12.0.226. Mono 5.16 has other issues under arm as well. https://github.com/Sonarr/Sonarr/issues/2769

[jeffkeller87]

Yep, mono 5.16.0.179 for me.

[beloware]

Confirm. Mono 5.16 ubuntu 16.04

An error occurred while processing feed.

Indexer works like before.

[sparten9999]

same issue here. on 5.16.0.179 as well

[metril]

Issue with mono 5.16.0.220 as well. I need to restart to resolve issues with indexers.

[jeffkeller87]

Are people having success with a cron job to restart sonarr? I have to restart the service and also "test" each indexer to clear the "unavailable" status before things can proceed as normal. Is this the case for anyone else? Is there a way to test indexers programmatically?

[markus101]

If you’re having this issue, downgrade mono to 5.14 or earlier. You don’t need to restart Sonarr and retest all indexers that way.

We’re leaving this issue open so it’s visible, but nothing we’re taking action on.

[lutzky]

markus101: Can the docker image be updated with a downgraded mono? (This is a great example of an advantage of running this in docker, that you wouldn't need to mess with the systemwide mono version)

[Gingey231]

markus101: Can the docker image be updated with a downgraded mono? (This is a great example of an advantage of running this in docker, that you wouldn't need to mess with the systemwide mono version)

This would be awesome if it could be implemented! or if someone knows the release where mono was updated that we can pull back down and use

[lutzky]

Seems it would be tricky. sonarr's Dockerfile is based on the Ubuntu Xenial with Mono Dockerfile, which installs the mono package from the official mono repository; no tags or anything. The built tags are just sequentially numbered, not indicating when mono was updated. I guess it's possible to binary-search through them and see what version of mono gets installed. The cronjob-restart works very well for me though, so I'm not very inclined to pin it to an old mono version. In fact, 5.16 appears to be the version marked in the xenial Release file, so an older version would require additional testing for compatibility with the xenial image... and one may as well do that with a newer version, who knows, maybe the bug is fixed by then.
It would be super helpful if we could get a reliable, quick way to reproduce the bug, rather than "wait a few days until a tracker drops out".

[Gingey231]

@lutzky I mean mine seems to be every hour or two, they all drop out! which is a pain, and i need to restart sonarr, and then retest them all manually!

[jeffkeller87]

Edit: The following only works for native GNU/Linux installs and _not_ for Docker installs of Sonarr.

For those are are unsure about how to downgrade mono like I was, I found this SO question helpful. You essentially pin your apt sources to a particular version in the mono repository by editing /etc/apt/sources.list.d/mono-official-stable.list

You can figure out the URL you need by walking the repo tree. Be sure to select a snapshots directory, the others will only have the latest version. I'm running Ubuntu 16.04 so the URL for the latest 5.14.x release is:

https://download.mono-project.com/repo/ubuntu/dists/stable-xenial/snapshots/5.14.0.177/main/

Which corresponds to an apt sources entry of:

deb http://download.mono-project.com/repo/ubuntu stable-xenial/snapshots/5.14.0.177 main

I then had to purge my way through a dependency nest before I could reinstall mono-complete, but it wasn't too bad.

[Gingey231]

For those are are unsure about how to downgrade mono like I was, I found this SO question helpful. You essentially pin your apt sources to a particular version in the mono repository by editing /etc/apt/sources.list.d/mono-official-stable.list

You can figure out the URL you need by walking the repo tree. Be sure to select a snapshots directory, the others will only have the latest version. I'm running Ubuntu 16.04 so the URL for the latest 5.14.x release is:

https://download.mono-project.com/repo/ubuntu/dists/stable-xenial/snapshots/5.14.0.177/main/

Which corresponds to an apt sources entry of:

deb http://download.mono-project.com/repo/ubuntu stable-xenial/snapshots/5.14.0.177 main

I then had to purge my way through a dependency nest before I could reinstall mono-complete, but it wasn't too bad.

That would be good if it used mono-complete in the docker image :p Managed to get 5.14 finally installed of mono-complete, but we shall see if it now works!

[Gingey231]

After downgrading using that methods, all i get is Error: Message not found, and sonarr just refuses to load sadly. Anyone have tips for downgrading it within the docker image?

[rourke]

I also have this problem with Sonarr version 2.0.0.5252 and Mono version 5.16.0.220. How would I be able to downgrade Mono if I use Sonarr (and Radarr) in Docker containers? Jeffkeller87's explanation would be for people not using the Docker container. I would prefer a way to keep Mono to v5.14 while still being able to pull new versions of Sonarr.

[Papsnoek]

So after 3 hours of trying to figure out how to downgrade Mono, I got it right, but it seems this did not resolve the issue?

Is there something else I might have missed?

[Papsnoek]

I noticed version 5.18 of Mono was released so I tried to update to latest.

So far so good.... been 2 days and my Indexers have not gone offline.

Not sure if someone else wants to try this and confirm?

[Gingey231]

If we see mono 5.18 working well for a week or so, would we be able to get this updated on the docker image at all please @markus101 ? :)

[Papsnoek]

Unfortunately mono 5.18 also has the same issue :( It seemed to work fine after updating for the first two days, but is now back to the same problem...

seeing version 5.14 did not work for me either, Im going to have to spend another few hours to try an older version and see if that works for me... Will report back when I find a working version.

[metril]

Mono 5.12 has been working fine for 2+ weeks now.

[ChromoX]

I feel like downgrading is not really fixing the problem.

There is a little more information about this bug in https://github.com/linuxserver/docker-sonarr/issues/84#issuecomment-453862187.

[Gingey231]

Cheers for the info @ChromoX however i don't seem to be getting the whole sluggish, bogged down experience, plus my issue happens daily, sometimes hourly, never fine for a few months! You might of been seeing a different issue possibly?

[Gh0stR1pp3r]

Reverted their mono base to 5.12, https://hub.docker.com/r/gh0str1pp3r/sonarr. Can confirm it works now for 2+ weeks.

[GarabatosRandom]

Same here, Mono 5.18 and it doesn't work after some time running, my crash log:

https://pastebin.com/6dgLeb3F

[Taloth]

Anyone having this issue with non-jackett setups? Coz I'm only seeing reports with docker+jackett.

[GarabatosRandom]

I don't use docker, just Sonarr in a fedora server with jackett

[Taloth]

Ok, questions:

• Your log contains various different hostnames for jackett but also a non-localhost IP, is this intended?
• Is it all jackett indexers that are failing?
• What's your exact mono version? (mono --version)
• What happens if you add another jackett indexer to Sonarr, does it produce the same timeout error during Test?
• What happens if you add rarbg directly to Sonarr, does it produce the same timeout error during Test?
• If you wget/curl that url as shown in the logs, does it return a result?
• What is 'some time'? Does it occur consistently within like 24h?
• What is the status of your ephemeral port range, how many ports are open/close_wait? (for example netstat -atnp)

[Buri]

Well, I had also this issue using docker image and in my case problem was docker was trying to use IPv6 for some reason. I disabled it with

sysctls:
      net.ipv6.conf.all.disable_ipv6: 1

and its working now. Hope that helps

[GarabatosRandom]

Your log contains various different hostnames for jackett but also a non-localhost IP, is this intended?

I changed all (in indexers) to my local lan ip, but dunno why my public ip (asus dns) is in the log.

Is it all jackett indexers that are failing?

yep

What's your exact mono version? (mono --version)

Mono JIT compiler version 5.18.1.0 (tarball Fri Mar 15 15:48:09 UTC 2019)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
TLS: __thread
SIGSEGV: altstack
Notifications: epoll
Architecture: amd64
Disabled: none
Misc: softdebug
Interpreter: yes
LLVM: yes(600)
Suspend: preemptive
GC: sgen (concurrent by default)

What happens if you add another jackett indexer to Sonarr, does it produce the same timeout error during Test?

It fails too.

What happens if you add rarbg directly to Sonarr, does it produce the same timeout error during Test?

It works.

If you wget/curl that url as shown in the logs, does it return a result?

Yes.

What is 'some time'? Does it occur consistently within like 24h?

Yep, something like 24h/36h, can't say exactly.

What is the status of your ephemeral port range, how many ports are open/close_wait? (for example netstat -atnp)

tcp 0 0 127.0.0.1:6789 127.0.0.1:35870 TIME_WAIT
tcp6 0 0 :::91* ::: LISTEN <-- jacket public port
tcp 0 0 192.168.1.4:8989 80.26.56.113:61132 ESTABLISHED
tcp 0 0 0.0.0.0:8989 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:7878 0.0.0.0:* LISTEN

[Taloth]

Thanks for the update, those are useful details.

Some further questions:

• If you test with 127.0.0.1 as the jackett ip instead of your local lan ip, does it fail or succeed? (Also check the exact error you get, it might be different than the timeout we saw before)
• If you run 'Test' in Sonarr, then while it's running, immediately run the netstat cmd again. Is there a new connection? If so, which.
• Does restarting Jackett remedy the issue as well? (opposed to restarting Sonarr, which is already known to remedy it)
• Is jackett actually listening on a non ipv6 port? It wasn't in the list you posted but I'd expect it to be there.

[GeoCookie]

Hello there,

I face the same problem regularly (every 1 or 2 day). Like the others, the only solution is to reboot sonarr and test all indexers.
I recently upgrade upgrade Sonarr to v3, but the problem remains.
If I try to use curl command on the url on the log file, it works.

New try of the day : I changed in the indexer's url the hostname by the ip address of jackett, and the indexer became available (without rebooting sonarr). That's a good point!
So I replaced on half of my indexers the hostname by the ip address of Jackett and i will see in the next few days if the other half becomes unavailables.

I will keep you inform

[simon816]

I appear to be having the same problem.

$ mono --version
Mono JIT compiler version 5.18.1.0 (tarball Fri Mar 15 20:41:32 UTC 2019)
$ lsb_release -d
Description:    Ubuntu 18.04.2 LTS

[Info] Bootstrap: Starting Sonarr - /opt/NzbDrone/NzbDrone.exe - Version 2.0.0.5322



md5-f22b2c2c0782150f86056fa4d7091094



$ sudo lsof -l -p 1056
COMMAND  PID     USER   FD   TYPE             DEVICE SIZE/OFF   NODE NAME
mono    1056     1001   18u  IPv6              50636      0t0    TCP localhost:36962->localhost:9117 (CLOSE_WAIT)
mono    1056     1001   19u  IPv6            1699986      0t0    TCP localhost:55218->localhost:9117 (CLOSE_WAIT)
mono    1056     1001   20u  IPv6              62160      0t0    TCP localhost:37104->localhost:9117 (CLOSE_WAIT)
mono    1056     1001   21u  IPv6            1135337      0t0    TCP localhost:48614->localhost:9117 (CLOSE_WAIT)
mono    1056     1001   22u  IPv6            1783262      0t0    TCP localhost:56096->localhost:9117 (CLOSE_WAIT)
mono    1056     1001   23u  IPv6             497257      0t0    TCP localhost:41840->localhost:9117 (CLOSE_WAIT)
mono    1056     1001   24u  IPv6            1804992      0t0    TCP localhost:56310->localhost:9117 (CLOSE_WAIT)
mono    1056     1001   25u  IPv6            1898615      0t0    TCP localhost:57260->localhost:9117 (CLOSE_WAIT)
mono    1056     1001   26u  IPv6            1920070      0t0    TCP localhost:57476->localhost:9117 (CLOSE_WAIT)
mono    1056     1001   27u  IPv6            1973150      0t0    TCP localhost:58028->localhost:9117 (CLOSE_WAIT)
mono    1056     1001   28u  IPv6            1995395      0t0    TCP localhost:58366->localhost:9117 (CLOSE_WAIT)
mono    1056     1001   29u  IPv6            2048567      0t0    TCP localhost:58906->localhost:9117 (CLOSE_WAIT)



md5-f921c8e9eb9d5fb12c4035318062fde7



mono    525     1001   30u  IPv4            3588777      0t0    TCP localhost:35688->localhost:9117 (CLOSE_WAIT)
mono    525     1001   31u  IPv4            3632193      0t0    TCP localhost:36488->localhost:9117 (CLOSE_WAIT)
mono    525     1001   32u  IPv4            3675775      0t0    TCP localhost:37068->localhost:9117 (CLOSE_WAIT)
mono    525     1001   33u  IPv4            3130924      0t0    TCP localhost:55632->localhost:9117 (CLOSE_WAIT)
mono    525     1001   34u  IPv4            3697484      0t0    TCP localhost:37272->localhost:9117 (CLOSE_WAIT)
mono    525     1001   35u  IPv4            3750221      0t0    TCP localhost:37802->localhost:9117 (CLOSE_WAIT)
mono    525     1001   36u  IPv4            3823253      0t0    TCP localhost:38520->localhost:9117 (CLOSE_WAIT)
mono    525     1001   37u  IPv4            3897197      0t0    TCP localhost:39470->localhost:9117 (CLOSE_WAIT)
mono    525     1001   38u  IPv4            3983162      0t0    TCP localhost:40888->localhost:9117 (CLOSE_WAIT)
mono    525     1001   39u  IPv4            4092150      0t0    TCP localhost:42764->localhost:9117 (CLOSE_WAIT)
mono    525     1001   40u  IPv4            4318596      0t0    TCP localhost:45940->localhost:9117 (CLOSE_WAIT)
mono    525     1001   41u  IPv4            4233086      0t0    TCP localhost:44982->localhost:9117 (CLOSE_WAIT)

[Panderine]

Same issue. Changing the jackett repo from ip to hostname and viceversa fixes the connection issue without restarting sonarr. When I get the error again I change back to ip or hostname and so forth

[memnos]

I have the same problem.
How do you change from ip to hostname?

BTW the strange is that i never had this problem with radarr.

[Panderine]

In settings->indexer-> Select the affected indexer -> in the url field change the hostname to ip address or viceversa.

For example:
http://jackett:9117/api/v2.0/indexers/1337x/results/torznab/

Change it to:
http://192.168.1.107:9117/api/v2.0/indexers/1337x/results/torznab/

[GeoCookie]

hi there,

Strange behavior for me that i didn't expect. Only the indexers using ip addresses goes offline (2 times this weekend). The others with the hostnames didn't have any problem.
I expect to have the opposite... :(

[mclssk]

Hi! Another with this problem! No Good news yet? Thx

[macbean101]

Just want to add another voice to this.

Version 3.0.1.449
Package Version 3.0.1
Mono Version 5.20.1.19

same issue. all jacket indexers fail until sonarr is restarted. radarr and other tools not affected.

key log entry:
[v3.0.1.449] System.Net.WebException: The operation has timed out.: 'http://192.168.123.195:9117/api/v2.0/indexers/kickasstorrent-kathow/results/torznab/api?t=tvsearch&cat=5030,5040&extended=1&apikey=(removed)&offset=0&limit=100' ---> System.Net.WebException: The operation has timed out.

however, if I replace (removed) with my apikey, the url works fine

[GeoCookie]

Hi there,

I am still testing some stuff, without success.

I tried to reverse my previous test

New try of the day : I changed in the indexer's url the hostname by the ip address of jackett, and the indexer became available (without rebooting sonarr). That's a good point!
So I replaced on half of my indexers the hostname by the ip address of Jackett and i will see in the next few days if the other half becomes unavailables.

Strange behavior for me that i didn't expect. Only the indexers using ip addresses goes offline (2 times this weekend). The others with the hostnames didn't have any problem.

Use hostnames on the first six indexers and IP addresses on the last six. Result : those with hostnames goes offline. Again, only the first six got problem. The problem seems not to be linked to the use of hostnames or IP addresses.

Tried to change the RSS sync to 30min : same problem

Change the network type of my sonarr container to host (it was on bridge) : same problem.

(current test) : move jackett to the same Docker host than sonarr and change its network mode to host too. This way, I can use “localhost” in the indexers URL. I have done the change this morning so I will see tomorrow if it’s change something.

[GeoCookie]

Hi there,

The problem seems to be fixed for me with the use of "localhost" instead of Jackett's hostname or IP address.
For people who use Docker containers like me, you have to change the network mode to "host" for the sonarr and jackett containers.

[Panderine]

When this happens to me If I curl the jackett url from sonarr's container console, I get normal results, but testing from within sonarr indexer settings results in a timeout.

[mehdiBezahaf]

Hi there,

The problem seems to be fixed for me with the use of "localhost" instead of Jackett's hostname or IP address.
For people who use Docker containers like me, you have to change the network mode to "host" for the sonarr and jackett containers.

This is not a permanent fix! I have been doing this now for ages, going back and forth between "localhost" and the Jackett's IP address! I mean I put "localhost" and it starts working for few days and it stops again working, so I switch to the IP address and it starts working again...then stops... it is probably one of the indexers sending some blank RageIDs that makes Sonarr crash!

[jshank]

Repeating the hacky workaround for docker until this is fixed

sudo crontab -e

   # m h  dom mon dow   command
   0 2 * * * /usr/local/bin/docker-compose -f /srv/docker-scripts/sonarr/docker-compose.yml restart sonarr

[pyrrhichios]

Hi,

Just wanted to chime in and say I have this exact issue, but I'm not using Docker, just standard Sonarr/Jackett installs.

Sonarr Version
2.0.0.5322
Mono Version
5.20.1.19

[Mansonjr]

Same issue here, Docker lastest version. whole indexers stopped until restart, on both programs, Radarr and Sonarr. Usenet still works, Torrent not.

Sonarr:
Version
2.0.0.5322
Mono Version
5.18.1.0
Radarr:
Version
0.2.0.1344
Mono Version
5.18.1.0 (tarball Fri Mar 15 20:45:47 UTC 2019)

[sataris]

Hi there,
The problem seems to be fixed for me with the use of "localhost" instead of Jackett's hostname or IP address.
For people who use Docker containers like me, you have to change the network mode to "host" for the sonarr and jackett containers.

This is not a permanent fix! I have been doing this now for ages, going back and forth between "localhost" and the Jackett's IP address! I mean I put "localhost" and it starts working for few days and it stops again working, so I switch to the IP address and it starts working again...then stops... it is probably one of the indexers sending some blank RageIDs that makes Sonarr crash!

This has been my solution for the past few months, disabling rarbg seemed to help (for a little while).

[sinister718]

I am also having the same issue via unraid docker

Version
2.0.0.5322
Mono Version
5.16.0

[SiemdeNijs]

same issue, ubuntu 18.04, mono 5.20.1.19, sonarr Ver. 2.0.0.5322

[shaboigan]

I have the same issue, not an occasional, I can almost set my watch to it.

Sonarr ver. 2.0.0.5322, Mono 5.18.0.240

[macbean101]

I did the same workaround as jshank above and haven't had this error since then.

0 23 * * * /usr/sbin/service sonarr restart

[lems111]

Same issue here on raspbian stretch, Sonarr Ver. 2.0.0.5322, mono version 5.20.1.19.
When I switched the IP address from 10.0.0.2 to 127.0.0.1 in my indexers, it resolved the issue.
Restarting sonarr service didnt seem to fix it.
I use OpenVPN, am wondering if that's causing an issue - I'll try to stop OpenVPN and see what happens. what do you guys think?

[jeffkeller87]

I use OpenVPN, am wondering if that's causing an issue - I'll try to stop OpenVPN and see what happens. what do you guys think?

I think it's been pretty well established that this is a mono bug. When I had this issue I was not running OpenVPN. Only rolling back to earlier versions of mono definitively fixed this for me.

[macbean101]

I just moved my sonarr from linux to windows and I haven't seen an instance
of this. Of course, i have OTHER issues now but not this one. (so far)

On Wed, May 29, 2019 at 2:19 PM Jeff Keller [e-mail]
wrote:

I use OpenVPN, am wondering if that's causing an issue - I'll try to stop
OpenVPN and see what happens. what do you guys think?

I think it's been pretty well established that this is a mono bug. When I
had this issue I was not running OpenVPN. Only rolling back to earlier
versions of mono definitively fixed this for me.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/Sonarr/Sonarr/issues/2802?email_source=notifications&email_token=AF722RLOH5NFWSFA6OYR5G3PXZYFNA5CNFSM4GB4ZOGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWPEP4I#issuecomment-496912369,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AF722RMMI6Y3YZXXKST556LPXZYFNANCNFSM4GB4ZOGA
.

[coolcat97]

I'm also having this issue. On docker.. Still haven't found a solution, has anyone else gotten the docker version working?

[zejar]

I'm having the same issue, it's been going on for a while now. I'm running the linuxserver/sonarr Docker image. When I restart Sonarr the indexers work for about half an hour and then stop.
The mono version I'm running is 5.20.1.19 and the Sonarr version is 2.0.0.5322.

[Falconne]

I have been using the docker image linuxserver/sonarr:5.14 as suggested in this comment for several days now and that has fixed the issue for me. For most of us I think this will work as a temporary solution, as that tag should always have the stable Sonarr version.

It's not a permanent fix as being stuck on an older Mono isn't ideal.

[Rhytox]

Just want to check in, same issue here.
Synology DS918+
Sonarr: 2.0.0.5322 (synocommunity package)
Mono: 5.18.0.240-12 (synocommunity package)

[coolcat97]

Have you tried the old linked docker image?

-
Thanks,
Zack
On Jun 10, 2019, 10:30 AM -0500, Rhytox [e-mail], wrote:

Just want to check in, same issue here.
Synology DS918+
Sonarr: 2.0.0.5322 (synocommunity package)
Jackett: 0.11.364.0 (docker)
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

[Rhytox]

Have you tried the old linked docker image? - Thanks, Zack
…
On Jun 10, 2019, 10:30 AM -0500, Rhytox @.*>, wrote: Just want to check in, same issue here. Synology DS918+ Sonarr: 2.0.0.5322 (synocommunity package) Jackett: 0.11.364.0 (docker) — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

The sonarr docker?
No as I am using the synocommunity package. I will have to replace everything with docker then.
If you are suggesting this then it will take some time as I don't have the time at the moment to switch to docker with everything :).

Regards

[coolcat97]

Yes. Docker is love. Docker is life.

-
Thanks,
Zack
On Jun 11, 2019, 8:36 AM -0500, Rhytox [e-mail], wrote:

Have you tried the old linked docker image? - Thanks, Zack
…
On Jun 10, 2019, 10:30 AM -0500, Rhytox @.*>, wrote: Just want to check in, same issue here. Synology DS918+ Sonarr: 2.0.0.5322 (synocommunity package) Jackett: 0.11.364.0 (docker) — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.
The sonarr docker?
No as I am using the synocommunity package. I will have to replace everything with docker then.
If you are suggesting this then it will take some time as I don't have the time at the moment to switch to docker with everything :).
Regards
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

[lems111]

Two items to share:

1. I have radarr running on the same local box and it hasn't run into the same problem which makes me question the mono root cause.
2. I noticed the following error in jackett when sonarr started to report unavailable for all indexers (maybe sonarr isn't handling this error properly?):
   Jun 29, 2019 03:28:44 AM Error

Jackett.Common.IndexerException: Exception (kickasstorrent): The operation was canceled. ---> System.OperationCanceledException: The operation was canceled.
at System.Net.Http.HttpClient.HandleFinishSendAsyncError(Exception e, CancellationTokenSource cts)
at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts) at Jackett.Common.Utils.Clients.HttpWebClient2NetCore.Run(WebRequest webRequest) in /home/appveyor/projects/jackett/src/Jackett.Common/Utils/Clients/HttpWebClient2NetCore.cs:line 241 at Jackett.Common.Utils.Clients.WebClient.GetString(WebRequest request) in /home/appveyor/projects/jackett/src/Jackett.Common/Utils/Clients/WebClient.cs:line 114 at Jackett.Common.Indexers.BaseWebIndexer.RequestStringWithCookies(String url, String cookieOverride, String referer, Dictionary2 headers) in /home/appveyor/projects/jackett/src/Jackett.Common/Indexers/BaseIndexer.cs:line 445
at Jackett.Common.Indexers.CardigannIndexer.PerformQuery(TorznabQuery query) in /home/appveyor/projects/jackett/src/Jackett.Common/Indexers/CardigannIndexer.cs:line 1230
at Jackett.Common.Indexers.BaseIndexer.ResultsForQuery(TorznabQuery query) in /home/appveyor/projects/jackett/src/Jackett.Common/Indexers/BaseIndexer.cs:line 331
--- End of inner exception stack trace ---
at Jackett.Common.Indexers.BaseIndexer.ResultsForQuery(TorznabQuery query) in /home/appveyor/projects/jackett/src/Jackett.Common/Indexers/BaseIndexer.cs:line 351
at Jackett.Common.Indexers.BaseWebIndexer.ResultsForQuery(TorznabQuery query) in /home/appveyor/projects/jackett/src/Jackett.Common/Indexers/BaseIndexer.cs:line 815
at Jackett.Server.Controllers.ResultsController.Torznab(TorznabRequest request) in /home/appveyor/projects/jackett/src/Jackett.Server/Controllers/ResultsController.cs:line 367

[Taloth]

@lems111 Talk to me on discord (or irc), link is in System->about. I'm not going to troubleshoot in a comment chain of me too posts.

[koto137]

Hi

I too have an issue with Sonarr dropping Jackes as not working indexers, I have found out that this happens when torrent with special character is passed to Sonarr "⭐". When searching manually and selecting such a result, the errors appear, but selecting any other search result without the start symbol, all working fine. There is plenty of examples at 1337x

[Taloth]

The underlying bug has been reported to the mono project team. (That's assuming it's the only one.)
In the meanwhile we've implemented a workaround for it.

I've rolled it out to the v2 develop (2.0.0.5337) and v3 phantom-develop (3.0.1.527) branches. The release should appear in System->Updates if you're on one of those two branches.
Once it's confirmed that there are no side-effects then the workaround will be rolled out to v2 master.

If you're going to test this change, please reply. Also disable the cronjob you may have added earlier.

---

The problem can be seen by checking the sonarr logs for 'timeout' warnings coming from Jackett supplied indexers. Such as Warn|Torznab|MyIndexer server is currently unavailable. http://localhost:9117/api/v2.0/indexers/MyIndexer/results/torznab/api?t=tvsearch&... The operation has timed out.
For each such timeout you'll find a new CLOSE_WAIT socket in netstat -anp going to jackett's port number, up to 12 per hostname ('localhost' or '127.0.0.1').
With the patched version the occasional timeout warning will still be logged, as they are legitimate timeouts where Jackett fails to respond within a given time period (probably 60 seconds). But the CLOSE_WAIT sockets should no longer occur.

[Buri]

Testing phantom-develop, will report results in few days

[dag420]

It fixed it for a while, but after a reboot they all started failing again.
Unable to connect to indexer: Error: ConnectFailure (No route to host): 'http://jackett:9117/api/v2.0/indexers/[removed]/results/torznab/api?t=tvsearch&cat=5000,5030,5040&extended=1&apikey=[removed]=0&limit=100'

System.Net.WebException: Error: ConnectFailure (No route to host): 'http://jackett:9117/api/v2.0/indexers/filelist/results/torznab/api?t=tvsearch&cat=5000,5030,5040,5060,5070&extended=1&apikey=q0uc5yw9f0pgtfnsko4t0306ay8vw9sd&offset=0&limit=100' ---> System.Net.WebException: Error: ConnectFailure (No route to host) ---> System.Net.Sockets.SocketException: No route to host at System.Net.Sockets.SocketAsyncResult.CheckIfThrowDelayedException () [0x00014] in :0 at System.Net.Sockets.Socket.EndConnect (System.IAsyncResult asyncResult) [0x0002c] in :0 at System.Net.Sockets.SocketTaskExtensions+<>c.b__2_1 (System.IAsyncResult asyncResult) [0x00006] in :0 at System.Threading.Tasks.TaskFactory1[TResult].FromAsyncCoreLogic (System.IAsyncResult iar, System.Func2[T,TResult] endFunction, System.Action1[T] endAction, System.Threading.Tasks.Task1[TResult] promise, System.Boolean requiresSynchronization) [0x00019] in <6649516e5b3542319fb262b421af0adb>:0 --- End of stack trace from previous location where exception was thrown --- at System.Net.WebConnection.Connect (System.Net.WebOperation operation, System.Threading.CancellationToken cancellationToken) [0x0019b] in :0 --- End of inner exception stack trace --- at System.Net.WebConnection.Connect (System.Net.WebOperation operation, System.Threading.CancellationToken cancellationToken) [0x00217] in :0 at System.Net.WebConnection.InitConnection (System.Net.WebOperation operation, System.Threading.CancellationToken cancellationToken) [0x000cc] in :0 at System.Net.WebOperation.Run () [0x0009a] in :0 at System.Net.WebCompletionSource1[T].WaitForCompletion () [0x00094] in <e8eb3d7a311640f484845e45cbec8973>:0 at System.Net.HttpWebRequest.RunWithTimeoutWorker[T] (System.Threading.Tasks.Task1[TResult] workerTask, System.Int32 timeout, System.Action abort, System.Func1[TResult] aborted, System.Threading.CancellationTokenSource cts) [0x000f8] in <e8eb3d7a311640f484845e45cbec8973>:0 at System.Net.HttpWebRequest.GetResponse () [0x00016] in <e8eb3d7a311640f484845e45cbec8973>:0 at NzbDrone.Common.Http.Dispatchers.ManagedHttpDispatcher.GetResponse (NzbDrone.Common.Http.HttpRequest request, System.Net.CookieContainer cookies) [0x0011b] in M:\BuildAgent\work\63739567f01dbcc2\src\NzbDrone.Common\Http\Dispatchers\ManagedHttpDispatcher.cs:81 --- End of inner exception stack trace --- at NzbDrone.Common.Http.Dispatchers.ManagedHttpDispatcher.GetResponse (NzbDrone.Common.Http.HttpRequest request, System.Net.CookieContainer cookies) [0x001b3] in M:\BuildAgent\work\63739567f01dbcc2\src\NzbDrone.Common\Http\Dispatchers\ManagedHttpDispatcher.cs:107 at NzbDrone.Common.Http.Dispatchers.FallbackHttpDispatcher.GetResponse (NzbDrone.Common.Http.HttpRequest request, System.Net.CookieContainer cookies) [0x000b5] in M:\BuildAgent\work\63739567f01dbcc2\src\NzbDrone.Common\Http\Dispatchers\FallbackHttpDispatcher.cs:53 at NzbDrone.Common.Http.HttpClient.ExecuteRequest (NzbDrone.Common.Http.HttpRequest request, System.Net.CookieContainer cookieContainer) [0x0007e] in M:\BuildAgent\work\63739567f01dbcc2\src\NzbDrone.Common\Http\HttpClient.cs:121 at NzbDrone.Common.Http.HttpClient.Execute (NzbDrone.Common.Http.HttpRequest request) [0x00008] in M:\BuildAgent\work\63739567f01dbcc2\src\NzbDrone.Common\Http\HttpClient.cs:57 at NzbDrone.Core.Indexers.HttpIndexerBase1[TSettings].FetchIndexerResponse (NzbDrone.Core.Indexers.IndexerRequest request) [0x0004b] in M:BuildAgentwork63739567f01dbcc2srcNzbDrone.CoreIndexersHttpIndexerBase.cs:321 at NzbDrone.Core.Indexers.HttpIndexerBase1[TSettings].FetchPage (NzbDrone.Core.Indexers.IndexerRequest request, NzbDrone.Core.Indexers.IParseIndexerResponse parser) [0x00000] in M:\BuildAgent\work\63739567f01dbcc2\src\NzbDrone.Core\Indexers\HttpIndexerBase.cs:298 at NzbDrone.Core.Indexers.HttpIndexerBase1[TSettings].TestConnection () [0x0000e] in M:BuildAgentwork63739567f01dbcc2srcNzbDrone.CoreIndexersHttpIndexerBase.cs:335`

[lems111]

@dag420 - the error doesn't look the same. The error that you posted is no route to host vs the timeout error.
Are you able to check the results of netstat -anp going to jackett's port number

[JackDallas]

The underlying bug has been reported to the mono project team. (That's assuming it's the only one.)
In the meanwhile we've implemented a workaround for it.

I've rolled it out to the v2 develop (2.0.0.5337) and v3 phantom-develop (3.0.1.527) branches. The release should appear in System->Updates if you're on one of those two branches.
Once it's confirmed that there are no side-effects then the workaround will be rolled out to v2 master.

If you're going to test this change, please reply. Also disable the cronjob you may have added earlier.

The problem can be seen by checking the sonarr logs for 'timeout' warnings coming from Jackett supplied indexers. Such as Warn|Torznab|MyIndexer server is currently unavailable. http://localhost:9117/api/v2.0/indexers/MyIndexer/results/torznab/api?t=tvsearch&... The operation has timed out.
For each such timeout you'll find a new CLOSE_WAIT socket in netstat -anp going to jackett's port number, up to 12 per hostname ('localhost' or '127.0.0.1').
With the patched version the occasional timeout warning will still be logged, as they are legitimate timeouts where Jackett fails to respond within a given time period (probably 60 seconds). But the CLOSE_WAIT sockets should no longer occur.

I'm on phantom-develop and have had this issue for a few months occurring every few day's, just updated my container I'll report back in a few days if it's fixed it for me :)

[JanJakes]

@Taloth Fixed it for me (on develop). Almost 3 days without a problem.

[Buri]

So far I also didn't notice any problems (phantom-develop)

[pyrrhichios]

2d 13 hrs uptime on this workaround release so far without issue. Seems promising!

[JackDallas]

3d 16h uptime also seems to be fixed for me, thanks a bunch @Taloth :D

[coolcat97]

Yeah, still having the same exact issues. will reoort in moening

@Taloth Fixed it for me (on develop). Almost 3 days without a problem.

How were you able to get it working? I updated to Dev and am still getting issues. Using docker.

[lems111]

No issues here. Happy as a clam on develop branch

[coolcat97]

Yeah, still having the same exact issues. will reoort in moening

@Taloth Fixed it for me (on develop). Almost 3 days without a problem.

How were you able to get it working? I updated to Dev and am still getting issues. Using docker.

No issues here. Happy as a clam on develop branch

Was able to also get it working on the Dev branch by grabbing latest image from docker on the 5.14 version as noted above!

Thank you!

[clajiness]

Working great on the v2 develop branch. No issues so far. Thanks!

[Taloth]

Thank you all for confirming the fix. I let it simmer for a whole, but I haven't heard anything bad.

Shipped it to v2 master, version 2.0.0.2338.
That way the linuxserver.io team can switch to 5.20 coz apparently there's another bug in mono 5.12 & 5.14 affecting Transmission. I swear, mono is lava with a few floating islands.

[elapse2039]

See same problem described in this thread on a native install of Sonar. Entire system was working fine up until a month ago running Mono 4. Upgraded to Mono 6 unintentionally, then back to 5 based on advice on this forum.

Currently running:

Ubuntu 18.04 deb http://apt.sonarr.tv/ master main
Sonar Version 2.0.0.5338
Mono Version 5.20.1.34 (deb https://download.mono-project.com/repo/ubuntu stable-bionic/snapshots/5.20.1.34 main)

[Taloth]

Maybe same error, not same problem. Please post on our forums for support. Once there, include all requested information such a debug level logs.

Do not reply here, I'll not troubleshoot new issues here.

[Thunder-Chief]

Reverted their mono base to 5.12, https://hub.docker.com/r/gh0str1pp3r/sonarr. Can confirm it works now for 2+ weeks.

This may be an obvious question - i'm a bit new here. -
If i change my docker image to yours will your image still continue to receive the updates that the official sonarr image does?

[Skwara]

I think I found a permanent solution (or at least I'm running Sonarr on mono 5.20.1.34 for 5 days with no issue where previously it occurred once or twice every day).

1. Create a new network in docker with the configuration below. What I did is I copied "bridge" network that I had by default and changed com.docker.network.driver.mtu to 9000. I also changed my subnet and gateway but I think MTU is key here.
2. Attach your containers only to this network. This step might not be necessary but I don't need both networks.
3. For indexer URL in Sonarr use Jackett's IP address from this network.

Network config:

[
    {
        "Name": "docker",
        "Id": "c88551fbda31432c65efd3c26cb6f2829dcfd5ca5b3e122e41b4df6e3eedb101",
        "Created": "2020-07-15T20:28:28.217628048+02:00",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "192.168.100.0/24",
                    "Gateway": "192.168.100.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Options": {
            "com.docker.network.bridge.default_bridge": "false",
            "com.docker.network.bridge.enable_icc": "true",
            "com.docker.network.bridge.enable_ip_masquerade": "true",
            "com.docker.network.bridge.host_binding_ipv4": "0.0.0.0",
            "com.docker.network.bridge.name": "docker1",
            "com.docker.network.driver.mtu": "9000"
        },
        "Labels": {}
    }
]