Sickchill: [APP SUBMITTED]: Unknown exception while loading URL http://thetvdb.com/api/F9C450E78D99172E/series/279536/all/en.zip: ChunkedEncodingError(ProtocolError("Connection broken: error(104, 'Connection reset by peer')", error(104, 'Connection reset by peer')),)

[Piedrosvs]

INFO

Python Version: 2.7.14 (default, May 28 2018, 14:46:53) [GCC 4.6.3]
Operating System: Linux-3.2.40-i686-with-glibc2.0
Locale: None
Branch: master
Commit: SickChill/SickChill@175df89cabf295358b8caa2d96dafadb2347622d
Link to Log: https://gist.github.com/3a6b87b3293e9784dca96810d9f704cf

ERROR

SHOWQUEUE-ADD :: [175df89] Unable to look up the show in /volume1/video/TV-Shows/Billions on theTVDB using ID 279536, not using the NFO. Delete .nfo and try adding manually again.: Unknown exception while loading URL http://thetvdb.com/api/F9C450E78D99172E/series/279536/all/en.zip: ChunkedEncodingError(ProtocolError("Connection broken: error(104, 'Connection reset by peer')", error(104, 'Connection reset by peer')),)

---

_STAFF NOTIFIED_: @SickChill/owners @SickChill/moderators

[Piedrosvs]

I've been having this problem for a while now. Recently moved from Sickrage (Echel0n) to SickChill. Problem existed already in SickRage. Remained in de migration to SickChill with the BenV spk and guide. Then completely removed SickChill and did a clean install and the problem remians. I get the error when adding a show in any form (new/existing). I also see that my existing shows haven't been updated since end of april.

From my PC i can load the mentioned page and download the zip file. From my NAS i'm also able to ping www.thetvdb.com, when i connect via Putty.
SickChill info:
Branch: master
Commit: 175df89cabf295358b8caa2d96dafadb2347622d
Version: v2019.05.15-3
Database Version: 44.0
Python Version: 2.7.14 (default, May 28 2018, 14:46:53) [GCC 4.6.3]
SSL Version: OpenSSL 1.0.2n 7 Dec 2017
OS: Linux-3.2.40-i686-with-glibc2.0
Locale: None.None

Already rebooted my WAN modem/router and NAS. I'm running out of ideas how to troubleshoot this.

[miigotu]

Your IP looks to be banned by thetvdb most likely, probably due to the spam echelons version does to providers.

[Piedrosvs]

It must be something different than a IP-block, because i can download the zip from my PC (on the same network) and TVDB also responds to a ping from my NAS where SickChill resides on. I already thought it might have something to do with my IP/portnumber combination?! That's why i already rebooted my WAN router/modem so that it gives SickChill different portnumbers (i hope). Or do you think TVDB recognizes the client (SickChill) another way? Any idea how i can fix this?
Thanks for your response.

[miigotu]

They track by User-Agent, API key, and IP. They also use cloudflare, which might be the reason this is failing.

@pro-src @CodyWoolaver this seems to have started happening for a few people just after the cfscrape update

[ghost]

The URL of the exception is plain HTTP without any kind of protection, Cloudflare or otherwise. The Cloudflare protection can be enabled for specific paths so I'm not doubting that the domain does have Cloudflare protection enabled for other parts of the site.

i.e. This works fine:
curl -O http://thetvdb.com/api/F9C450E78D99172E/series/279536/all/en.zip

I'm guessing that the server is to blame for this error: https://github.com/kennethreitz/requests/issues/4771
And in conclusion: https://github.com/kennethreitz/requests/issues/4484#issuecomment-360982818

@miigotu Maybe detect this specific failure and retry the request?

[Piedrosvs]

I can try to make a wireshark trace of the session if this helps. Please let me know.

[miigotu]

@pro-src the tvdb api has changed and we were supposed to rewrite the lib to use a new version a long time ago. Their server code is custom to the point where sometimes it intentionally drops connections seemingly at random (not adhering to http standards) just because too many people are hitting the api at one time using the same application key. I think the whole issue goes away if we rewrite the lib that accesses tvdb, but it was cumbersome and we had planned to add tvmaze first. Trouble is that I dont have tons of time these days and there hasnt been a whole lot of help since most of the team has forked and I haven't been around to stoke up new contributions much.

Lets see if this problem sorts itself out, but Im curious how often it happens? The same show would update again shortly after this error happened anyways, because the update didnt succeed.

Seeing as this isnt a mass reported thing, this might be limited to a very small base of users. Possibly due to very slow connection (maybe when torrent client is using all the bandwidth) and the server is killing the connection when it takes too long.

[ghost]

@miigotu Good stuff but I can't reproduce it.

Code snippet

import sys
import socket
import time

try: # Python 2
    import httplib
except:
    import http.client as httplib

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('thetvdb.com', 80))

msg = (
    'GET /api/F9C450E78D99172E/series/279536/all/en.zip HTTP/1.1',
    'Host: thetvdb.com:80',
#    'Connection: keep-alive', # <-- requests default
#    'Accept: */*', # <-- default for python, requests, and curl
#    'Accept-Encoding: gzip, deflate', # <-- requests default
#    'Accept-Encoding: identity', # <-- python default
#    'User-Agent: python-requests/2.21.0', # <-- requests default
#    'User-Agent: curl/7.64.0', # <-- curl default
)

msg = ('\r\n'.join(msg) + '\r\n\r\n').encode()

s.send(msg)
s.settimeout(0.5)

# Avoid reading to much of the body so we can wait around
httplib._MAXLINE = 1024
# The response is automatically buffered when using Python 3
# Use Python 2 to prevent buffering when using this class
response = httplib.HTTPResponse(s, debuglevel=1)
response.begin()

while True:
     sys.stdout.write('.')
     sys.stdout.flush()

     if not response.read(1024):
         break

     time.sleep(5)

sys.stdout.write('\n');

The only way that I can generate that error is If I make the server wait when sending the request. i.e. Leave out the end-of-headers delimiter \r\n when sending the request. I agree it could have something to do with load balancing but then how could we fix it? What changes need to be made to the tvdb library?

@Piedrosvs I'd like to see the wireshark output, although I don't know that we'll see anything interesting there.

The error is initially raised by the socket: error(104, 'Connection reset by peer') then as httplib.IncompleteRead but is caught and raised by urllib3 and requests as their own errors respectively.

Also, based on the response headers, we're hitting Cloudflare's cache.

[Piedrosvs]

WireShark trace SickChill#5479.zip

[Piedrosvs]

There are 2 traces. The first is of the add-show after selecting the show (no2 Pick a Show) and hitting "Add show".
The second trace is of the complete process of adding.
Hope this helps out.

[ghost]

@Piedrosvs

1. https://packettotal.com/app/analysis?id=09f5f97e819b9fe005483911d79c6638&name=http
   
   Headers

GET /api/F9C450E78D99172E/series/359913/all/en.zip HTTP/1.1 Host: thetvdb.com Connection: keep-alive Accept-Encoding: gzip, deflate Accept: */* User-Agent: python-requests/2.20.0 Cookie: __cfduid=db5f6ede44a86c2c466f4f419222b3a2e1557956359 HTTP/1.1 200 OK Date: Wed, 15 May 2019 21:39:19 GMT Content-Type: application/zip Content-Length: 4168 Connection: keep-alive Last-Modified: Sat, 11 May 2019 17:01:38 GMT ETag: "5cd6fff2-1048" Expires: Thu, 16 May 2019 01:39:19 GMT Cache-Control: public, max-age=14400 CF-Cache-Status: HIT Accept-Ranges: bytes Vary: Accept-Encoding Server: cloudflare CF-RAY: 4d78438c6c109c15-AMS

/api/F9C450E78D99172E/series/359913/all/en.zip
Expected: 4168
Actual: 4168

1. https://packettotal.com/app/analysis?id=607eead715ea4e576645b1680a42b9b2&name=http

Headers

GET /api/F9C450E78D99172E/series/279536/all/en.zip HTTP/1.1 Host: thetvdb.com Connection: keep-alive Accept-Encoding: gzip, deflate Accept: */* User-Agent: python-requests/2.20.0 Cookie: __cfduid=d8e2c900d939bcb6d86050321a20922081558644662 HTTP/1.1 200 OK Date: Thu, 23 May 2019 20:51:03 GMT Content-Type: application/zip Content-Length: 15023 Connection: keep-alive Last-Modified: Tue, 14 May 2019 10:25:49 GMT ETag: "5cda97ad-3aaf" Expires: Fri, 24 May 2019 00:51:03 GMT Cache-Control: public, max-age=14400 CF-Cache-Status: REVALIDATED Accept-Ranges: bytes Vary: Accept-Encoding Server: cloudflare CF-RAY: 4db9e7d999789cbd-AMS

/api/F9C450E78D99172E/series/279536/all/en.zip
Expected: 15023
Actual: 14194

Interesting...

The server is sending the expected data which can be seen by reconstructing the partial zip file. The one that I examined had the "en.xml" completely intact but was missing the "banners.xml". It's just closing the connection before sending the entire zip file for whatever reason.

The server sends 4 TCP reset packets followed by the client's TCP ACK for which the server responds with a fifth TCP reset packet. Maybe the server thinks that your end is messing up? This could be caused by a number of things. The HTTP transfer is not chunked. I'm not sure that this is the server's fault at all.

If this is consistently reproducible on your end(every attempt) then I'm going to guess that there is something unique to your network causing the server to become confused.

Are you doing anything else on your system when this occurs? Other non-system processes, etc?
Does this always happen?

Could you try from another network and/or device?
Can you run the python code snippet from my previous post on the affected system and a non-affected system on the same network?

[miigotu]

Look at the sender port on https://packettotal.com/app/analysis?id=607eead715ea4e576645b1680a42b9b2&name=http
I don't understand why the port is switching?

there is a __cfduid cookie sent on the zip request but not the xml request

[ghost]

The port is changing because the socket connection isn't being reused. The server doesn't explicitly close the previous requests, it replies with keep-alive. The requests/urllib3 default number of connection pools is 10 and the max number of connections per pool is only one. The connection pooling appears to be working for the last two requests in that capture.

The cookie is inconsequential based on my tests.

[miigotu]

Its a strange issue. going by the cookie though I would assume its failing right after triggering CF.

[Piedrosvs]

If this is consistently reproducible on your end(every attempt) then I'm going to guess that there is something unique to your network causing the server to become confused.

Yes, this happens every time when i add any show, for a while now. Somewhere end of april. Also shows don't get upated anymore in SickChill

Are you doing anything else on your system when this occurs? Other non-system processes, etc?

Yes, I am runnig more applications like CouchPotato, Spotweb, SabNZBD, DNS server etc. The packet trace was done by a Meraki MX apliance in between, capturing any traffic between my NAS and theTVDB.com

Does this always happen?

Yes.

Could you try from another network and/or device?

When i try to open the url from any PC in my network (behind the same Meraki) it downloads OK.

Can you run the python code snippet from my previous post on the affected system and a non-affected system on the same network?

I only have one NAS, so i wouldn't know how to do this. Any tips?

[Piedrosvs]

I can try running your snippet from my NAS. I'm kind of a noob in Linux, so may take me som work/time. Gues just paste in a shell script, make executable and run (as root?)?
Know i think of it, i could also try to run it from my Raspberry Pi (Kodi player). That is Linux also.
Will try something out this weekend.

[Piedrosvs]

Look at the sender port on https://packettotal.com/app/analysis?id=607eead715ea4e576645b1680a42b9b2&name=http
I don't understand why the port is switching?

This might have to do with the fact that this are 2 sepeate sessions? When i fill in the series name in SickChill, it first will look it up to TheTVDB.com (first portnumber?). Then when i select the right series in the Gui and click "Add Show" it wil try to download the actual series info from TheTVDB.com (second portnumber?)
I'm just guessing, but maybe that's it?

[ghost]

The snippet can be executed without root privileges and you could paste it into the python interpreter over SSH or make an executable. Run with Python 2.

I only have one NAS, so i wouldn't know how to do this. Any tips?

Installing SickChill on another system on the same network should work.

This might have to do with the fact that this are 2 sepeate sessions?

I think they would be using the same requests session but if not then, yes. Really depends on what you mean by session though i.e. there was definitely two different socket connections made during the second capture.

[Piedrosvs]

Here's the output of the script, run from my NAS:
Pieter@DS214play:~$ /usr/local/python/bin/python ./pro-src_snippet
reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Sat, 25 May 2019 15:54:09 GMT
header: Content-Type: application/zip
header: Content-Length: 15023
header: Connection: keep-alive
header: Set-Cookie: __cfduid=d167962cbe230ad7773181cf9dc4c40e51558799648; expires=Sun, 24-May-20 15:54:08 GMT; path=/; domain=.thetvdb.com; HttpOnly
header: Last-Modified: Tue, 14 May 2019 10:25:49 GMT
header: ETag: "5cda97ad-3aaf"
header: Expires: Sat, 25 May 2019 19:54:09 GMT
header: Cache-Control: public, max-age=14400
header: CF-Cache-Status: REVALIDATED
header: Accept-Ranges: bytes
header: Server: cloudflare
header: CF-RAY: 4dc8afaddb099d4e-AMS
..............Traceback (most recent call last):
File "./pro-src_snippet", line 40, in
if not response.read(1024):
File "/usr/local/python/lib/python2.7/httplib.py", line 597, in read
s = self.fp.read(amt)
File "/usr/local/python/lib/python2.7/socket.py", line 384, in read
data = self._sock.recv(left)
socket.error: [Errno 104] Connection reset by peer

[Piedrosvs]

And the curl command:

curl -O http://thetvdb.com/api/F9C450E78D99172E/series/279536/all/en.zip
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
93 15023 93 14083 0 0 460k 0 --:--:-- --:--:-- --:--:-- 474k
curl: (56) Recv failure: Connection reset by peer

[Piedrosvs]

Curl command from Rapsberry Pi (LibreElec):

#

LibreELEC

https://libreelec.tv

#

LibreELEC (official): 8.2.5 (RPi2.arm)
LibreELEC:~ # curl -O http://thetvdb.com/api/F9C450E78D99172E/series/279536/all/
en.zip
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
93 15023 93 14075 0 0 2418 0 0:00:06 0:00:05 0:00:01 3268
curl: (56) Recv failure: Connection reset by peer
LibreELEC:~ #

[Piedrosvs]

The snippet from my Raspberry Pi (Kodi):
LibreELEC:~ # python ./pro-src_snippet
reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Sat, 25 May 2019 16:38:36 GMT
header: Content-Type: application/zip
header: Content-Length: 15023
header: Connection: keep-alive
header: Set-Cookie: __cfduid=d903a4fca683a238f44a5736d0af007f01558802316; expires=Sun, 24-May-20 16:38:36 GMT; path=/; domain=.thetvdb.com; HttpOnly
header: Last-Modified: Tue, 14 May 2019 10:25:49 GMT
header: ETag: "5cda97ad-3aaf"
header: Expires: Sat, 25 May 2019 20:38:36 GMT
header: Cache-Control: public, max-age=14400
header: CF-Cache-Status: HIT
header: Accept-Ranges: bytes
header: Server: cloudflare
header: CF-RAY: 4dc8f0cd6c4ebda5-AMS
..............Traceback (most recent call last):
File "./pro-src_snippet", line 40, in
if not response.read(1024):
File "/usr/lib/python2.7/httplib.py", line 597, in read
File "/usr/lib/python2.7/socket.py", line 384, in read
socket.error: [Errno 104] Connection reset by peer
LibreELEC:~ #

[Piedrosvs]

Like i said. Downloading from any Windws PC in the network from my browser works fine. Though running curl from my windows PC also gives a connection reset:
C:\Users\Pieter>curl -O http://thetvdb.com/api/F9C450E78D99172E/series/279536/all/en.zip
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
6 15023 6 943 0 0 943 0 0:00:15 0:00:01 0:00:14 861
curl: (56) Recv failure: Connection was reset

[Piedrosvs]

I also installed Python 2.7.16 on my Windows PC and ran your snippet:
Python 2.7.16 (v2.7.16:413a49145e, Mar 4 2019, 01:37:19) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license()" for more information.

=========== RESTART: D:/Program Files/Python2.7/pro-src_snippet.py ===========
reply: 'HTTP/1.1 200 OK\r\n'

Traceback (most recent call last):
File "D:/Program Files/Python2.7/pro-src_snippet.py", line 34, in
response.begin()
File "D:\Program Files\Python2.7\lib\httplib.py", line 470, in begin
self.msg = HTTPMessage(self.fp, 0)
File "D:\Program Files\Python2.7\lib\mimetools.py", line 25, in __init__
rfc822.Message.__init__(self, fp, seekable)
File "D:\Program Files\Python2.7\lib\rfc822.py", line 108, in __init__
self.readheaders()
File "D:\Program Files\Python2.7\lib\httplib.py", line 316, in readheaders
line = self.fp.readline(_MAXLINE + 1)
File "D:\Program Files\Python2.7\lib\socket.py", line 480, in readline
data = self._sock.recv(self._rbufsize)
error: [Errno 10054] De externe host heeft een verbinding verbroken

The last line is Dutch (my Windows is Dutch) and means: External host has terminated a connection

[Piedrosvs]

Could installing SickChill for Windos help on resolving this issue? I can try if you like.

[ghost]

@Piedrosvs Thanks for running those tests. The web browser is probably retrying with a range request to fetch the remaining part of the file when that error occurs. At least I seem to recall browsers doing something of the sort.

If this isn't an IP configuration issue on your end, it's going to be a problem with a server in-between you and Cloudflare unless (highly unlikely) Cloudflare is responsible. The problem occurs whenever you hit Cloudflare's cache as well, meaning that http://thetvdb.com isn't being contacted in that case and you're just talking to Cloudflare.

I would perform the following:

• ensure that I have proper IP configuration.
• trace the IP packet route and try to identify a problem there, tracert(Windows) or traceroute(Linux) shell commands

One way to verify the problem isn't on your end, is for us to use the same proxy. If it only happens on your end when using the same proxy, it is confirmed.

[ghost]

@Piedrosvs Try this on your windows PC that had the same problem with curl:
curl -Sv --proxy 'http://35.246.211.162:80' http://thetvdb.com/api/F9C450E78D99172E/series/279536/all/en.zip -O

[Piedrosvs]

It looks like i'm having trouble with the proxy:
C:\Users\Pieter>curl -Sv --proxy 'http://35.246.211.162:80' http://thetvdb.com/api/F9C450E78D99172E/series/279536/all/en.zip -O

• Unsupported proxy scheme for ''http://35.246.211.162:80''
• Closing connection -1
  curl: (7) Unsupported proxy scheme for ''http://35.246.211.162:80''

C:\Users\Pieter>curl -Sv --proxy http://35.246.211.162:80 http://thetvdb.com/api/F9C450E78D99172E/series/279536/all/en.zip -O

• Trying 35.246.211.162...
• TCP_NODELAY set
  % Total % Received % Xferd Average Speed Time Time Time Current
  Dload Upload Total Spent Left Speed
  0 0 0 0 0 0 0 0 --:--:-- 0:00:20 --:--:-- 0* connect to 35.246.211.162 port 80 failed: Timed out
• Failed to connect to 35.246.211.162 port 80: Timed out
• Closing connection 0
  curl: (7) Failed to connect to 35.246.211.162 port 80: Timed out

C:\Users\Pieter>

[Piedrosvs]

I do seem to reach thetvdb.com OK:
C:\Users\Pieter>tracert www.thetvdb.com

Tracing route to www.thetvdb.com [104.16.228.14]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms 192.168.53.254
2 <1 ms 1 ms 1 ms static.kpn.net [195.190.228.17]
3 * * * Request timed out.
4 7 ms 6 ms 6 ms cloudflare.telecity2.nl-ix.net [193.239.117.114]
5 5 ms 5 ms 5 ms 104.16.228.14

Trace complete.

[Piedrosvs]

I'm wondering if this can have anything to do with http / https. When i type "thetvdb.com/api/F9C450E78D99172E/series/279536/all/en.zip" on my laptop from work, it doesnt work. When i then type "thetvdb.com" it connects but the url shows the https address. My work PC has Kaspersky on it, which i have no control over (corparate), and which is suspect to blok non-SSL connections sometimes. I see that SickChill also uses http instead of https.......... Can that be related?

[ghost]

That random proxy stopped working... Here is a list if you want to experiment: https://www.us-proxy.org

[Piedrosvs]

C:\Users\Pieter>curl -Sv --proxy http://54.218.1.67:80 http://thetvdb.com/api/F9C450E78D99172E/series/279536/all/en.zip -O

• Trying 54.218.1.67...
• TCP_NODELAY set
  % Total % Received % Xferd Average Speed Time Time Time Current
  Dload Upload Total Spent Left Speed
  0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 54.218.1.67 (54.218.1.67) port 80 (#0)

GET http://thetvdb.com/api/F9C450E78D99172E/series/279536/all/en.zip HTTP/1.1
Host: thetvdb.com
User-Agent: curl/7.55.1
Accept: /
Proxy-Connection: Keep-Alive

0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0< HTTP/1.1 200 OK
< Date: Mon, 27 May 2019 07:44:14 GMT
< Server: cloudflare
< Content-Type: application/zip
< Content-Length: 15055
< Last-Modified: Sun, 26 May 2019 08:21:36 GMT
< ETag: "5cea4c90-3acf"
< Expires: Mon, 27 May 2019 11:44:14 GMT
< Cache-Control: public, max-age=14400
< CF-Cache-Status: HIT
< Accept-Ranges: bytes
< CF-RAY: 4dd65ccd1bc72a77-SEA
< Set-Cookie: __cfduid=db3cf8ebeddf9dc7e0774e9b8f8e305b21558943054; expires=Tue, 26-May-20 07:44:14 GMT; path=/; domain=.thetvdb.com; HttpOnly
<
{ [967 bytes data]

• Recv failure: Connection was reset
  93 15055 93 14032 0 0 14032 0 0:00:01 0:00:01 --:--:-- 7609
• Closing connection 0
  curl: (56) Recv failure: Connection was reset

[Piedrosvs]

I've tried several. The available ones all had the same problem. I guess this means troubleshooting my IP/NAT config?!

[ghost]

Unfortunately, yes, the problem does appear to be only on your end. Please let us know if you figure something out.

curl -Sv --proxy http://54.218.1.67:80 http://thetvdb.com/api/F9C450E78D99172E/series/279536/all/en.zip -O

Works just fine for me.

[ghost]

@miigotu Unless I'm missing something, I think it's safe to close this issue. It doesn't appear to be pertinent to SickChill.

[Piedrosvs]

Will do so. If i find out anything else, i will let it know.

Thank you very much for your help and patience!