Biopython: urlopen error EOF occurred in violation of protocol (_ssl.c:777)

[Achllle]

Setup

I encountered an error while running code that worked fine one week ago (I don't think I changed anything since). It seems like an error with OpenSSL/urllib but a quick search online can't solve my problem so figured posting here would be of most help.

Biopython version, Python version, and operating system as follows:

1.70
3.6.4 |Anaconda, Inc.| (default, Jan 16 2018, 18:10:19) 
CPython
Linux-4.13.0-37-generic-x86_64-with-debian-stretch-sid

Error message

Traceback (most recent call last):
  File "/home/achille/miniconda3/lib/python3.6/urllib/request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/home/achille/miniconda3/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/achille/miniconda3/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/achille/miniconda3/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/achille/miniconda3/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/achille/miniconda3/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/home/achille/miniconda3/lib/python3.6/http/client.py", line 1400, in connect
    server_hostname=server_hostname)
  File "/home/achille/miniconda3/lib/python3.6/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/home/achille/miniconda3/lib/python3.6/ssl.py", line 814, in __init__
    self.do_handshake()
  File "/home/achille/miniconda3/lib/python3.6/ssl.py", line 1068, in do_handshake
    self._sslobj.do_handshake()
  File "/home/achille/miniconda3/lib/python3.6/ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:777)

Steps to reproduce

from Bio import Entrez
Entrez.email = '...'
handle = Entrez.esearch(db='pubmed', sort='relevance', retmax='20', retmode='xml', term='fever')

[Achllle]

Restarting Ubuntu worked, not sure how exactly. Closing for now.

[peterjc]

This is not a Biopython problem, it is something within the standard libraries of Python itself - possibly due to a temporary problem at the NCBI. The fact that this has gone away on a reboot suggests a temporary network glitch somewhere.

If you're interested, we use HTTPS to connect to the NCBI, and Python is using an SSL library to verify the certificate for the NCBI website. The actual URL here is:

>>> handle.url
'https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=pubmed&term=fever&sort=relevance&retmax=20&retmode=xml&tool=biopython'

The example worked for me just now. You can also try https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=pubmed&term=fever&sort=relevance&retmax=20&retmode=xml or similar in your browser and it should work - and it doesn't that suggest something odd about your local network.

I found lots of similar unanswered questions on StackExchange etc. The most informative issue I could find suggests one known way to trigger this exception is with an old insecure web-server and a modern client library, where they have no encryption ciphers in common. I doubt that applied here (especially as the problem went away on its own).

Anyway, I'm glad to hear it is working again for you now :)

[Achllle]

Cool, thanks for the insight! That's what I thought but I'm still learning about APIs and connections so wasn't sure. The specific error code (_ssl.c:777) didn't yield any results on google either. I tried running it dozens of times which made it seem it wasn't a temporary connection issue. Weird. Good thing the good ol' 'have your tried turning it off and on again?' did the trick :smile:

[philshem]

According to this post

https://github.com/kennethreitz/requests/issues/3006#issuecomment-274058323

it can be fixed on python 3 by installing the following packages:

pip install pyopenssl ndg-httpsclient pyasn1 urllib3

(testing now with biopython, although this is the kind of error you get after the script runs for a long long time...)

[philshem]

my workaround is now this function

def get_entrez(handle):

    results = None
    waiter = 0
    while results is None and waiter < 10:
        results = Entrez.read(handle)
        time.sleep(waiter)
        waiter += 1

    return results

[srilekhajagarlamudi98]

According to this post

psf/requests#3006 (comment)

it can be fixed on python 3 by installing the following packages:

pip install pyopenssl ndg-httpsclient pyasn1 urllib3

(testing now with biopython, although this is the kind of error you get after the script runs for a long long time...)

tq u saved my time..