Docker-py: Timeout incompatibility

[ghost]

Hi, i'm on a Debian Sid with docker-py 0.5.3-1 and i have a problem with timeout parameter :

import docker

c = docker.Client(base_url='unix://var/run/docker.sock', version='1.15', timeout=2.0)
l = c.containers(all=True)      

trace :

Traceback (most recent call last):                                                                                 
  File "./check.py", line 10, in <module>                                                                          
    l = c.containers(all=True)                                                                                     
  File "/usr/lib/python2.7/dist-packages/docker/client.py", line 493, in containers                                
    res = self._result(self._get(u, params=params), True)                                                          
  File "/usr/lib/python2.7/dist-packages/docker/client.py", line 76, in _get                                       
    return self.get(url, **self._set_request_timeout(kwargs))                                                      
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 469, in get                                   
    return self.request('GET', url, **kwargs)                                                                      
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 457, in request                               
    resp = self.send(prep, **send_kwargs)                                                                          
  File "/usr/lib/python2.7/dist-packages/requests/sessions.py", line 569, in send                                  
    r = adapter.send(request, **kwargs)                                                                            
  File "/usr/lib/python2.7/dist-packages/requests/adapters.py", line 362, in send                                  
    timeout=timeout                                                                                                
  File "/usr/lib/python2.7/dist-packages/requests/packages/urllib3/connectionpool.py", line 522, in urlopen        
    body=body, headers=headers)                                                                                    
  File "/usr/lib/python2.7/dist-packages/requests/packages/urllib3/connectionpool.py", line 305, in _make_request  
    timeout_obj = self._get_timeout(timeout)                                                                       
  File "/usr/lib/python2.7/dist-packages/requests/packages/urllib3/connectionpool.py", line 285, in _get_timeout   
    return Timeout.from_float(timeout)                                                                             
  File "/usr/lib/python2.7/dist-packages/requests/packages/urllib3/util/timeout.py", line 152, in from_float       
    return Timeout(read=timeout, connect=timeout)                                                                  
  File "/usr/lib/python2.7/dist-packages/requests/packages/urllib3/util/timeout.py", line 95, in __init__          
    self._connect = self._validate_timeout(connect, 'connect')                                                     
  File "/usr/lib/python2.7/dist-packages/requests/packages/urllib3/util/timeout.py", line 125, in _validate_timeout
    "int or float." % (name, value))                                                                               
ValueError: Timeout value connect was Timeout(connect=2.0, read=2.0, total=None), but it must be an int or float.  

To fix the problem i have to modify directly /usr/lib/python2.7/dist-packages/requests/packages/urllib3/connectionpool.py

def _get_timeout(self, timeout):                                            
    """ Helper that always returns a :class:`urllib3.util.Timeout` """      
    if timeout is _Default:                                                           
        return self.timeout.clone()                                         

    if isinstance(timeout, Timeout): <========================== Timeout is not a urllib3 timeout
        return timeout.clone()                                              
    else:                                                                   
        # User passed us an int/float. This is for backwards compatibility, 
        # can be removed later                                                                                                 
        return Timeout.from_float(timeout._connect ) <======================= manually entered _connect

I thnk it's a docker-py problem...

[dims]

Which version of requests?

[ghost]

requests 2.4.3-2

[kuba]

FYI this has been reported in Debian as #767445 and IMO it is a python-docker packaging problem, which I explain in the comment.

[iTech-Developer]

I do not think it is python-docker packaging problem, because when I install docker-py with pip install docker-py on debian Wheezy/Jessie I get the exact same problem.

Modifying the _requests_ library is not a solution for me because it can break other libraries. So I changed _docker-py_ as follows:

Replaced in /docker/unixconn/unixconn.py (usually under _/usr/local/lib/python2.7/dist-packages_):

try:
    import requests.packages.urllib3.connectionpool as connectionpool
except ImportError:
    import urllib3.connectionpool as connectionpool

With just import urllib3.connectionpool as connectionpool

And similarly I replaced in /docker/ssladapter/ssladapter.py:

try:
   import requests.packages.urllib3 as urllib3
except ImportError:
   import urllib3

With import urllib3

Same thing with from requests.packages.urllib3._collections import RecentlyUsedContainer

[kuba]

FYI Debian Python Modules Packaging Team is currently working to fix that. It is python-requests that is broken, and will hopefully be fixed soon :)

[inducer]

Appears fixed in Debian's python-requests version 2.4.3-4.

[voronaff]

Hi. It seems to me I have a similar problem with timeout.
I try to commit a container

c = Client(base_url="tcp://%s:%s" % (host, port), version="auto", timeout=300)
c.commit(container_Id, new_image_name, new_tag)

but I get error

File "/usr/local/lib/python2.7/dist-packages/docker/client.py", line 70, in __init__
    self._version = self._retrieve_server_version()
  File "/usr/local/lib/python2.7/dist-packages/docker/client.py", line 90, in _retrieve_server_version
    'Error while fetching server API version: {0}'.format(e)
docker.errors.DockerException: Error while fetching server API version: a float is required

[shin-]

pip freeze output? You may have outdated packages.

[voronaff]

maybe...
pip freeze output:

adium-theme-ubuntu==0.3.4
ansible==1.7.2
apt-xapian-index==0.46
backports.ssl-match-hostname==3.4.0.2
boto==2.36.0
characteristic==0.1.0
chardet==2.2.1
cm-api==9.0.0
colorama==0.3.1
command-not-found==0.3
debtagshw==0.1
defer==1.0.6
dirspec==13.10
docker-py==1.1.0
duplicity==0.6.23
ecdsa==0.13
html5lib==0.999
httplib2==0.9
idna==0.9
Jinja2==2.7.3
lockfile==0.8
lxml==3.3.6
MarkupSafe==0.23
oauthlib==0.6.1
oneconf==0.3.7
PAM==0.4.2
paramiko==1.15.2
pexpect==3.2
Pillow==2.6.1
piston-mini-client==0.7.5
pyasn1==0.1.7
pyasn1-modules==0.0.5
pycrypto==2.6.1
pycups==1.9.67
pygobject==3.14.0
pyOpenSSL==0.13.1
pyserial==2.6
python-apt===0.9.3.10ubuntu1
python-debian==0.1.22
pyxdg==0.25
PyYAML==3.11
reportlab==3.1.8
requests==2.6.0
service-identity==1.0.0
sessioninstaller==0.0.0
six==1.9.0
software-center-aptd-plugins==0.0.0
system-service==0.1.6
Twisted-Core==14.0.2
Twisted-Web==14.0.2
unity-lens-photos==1.0
urllib3==1.8.3
virtualenv==12.0.7
websocket-client==0.29.0
wheel==0.24.0
xdiagnose==3.6.6
zope.interface==4.1.1

[voronaff]

Hi guys!
Do you have any news about problem?

[shin-]

What happens if you pip install -U requests==2.5.3? Same issue?

[shin-]

Ah nevermind, this is something else...

[shin-]

No yeah, actually, try downgrading your version of requests (pip install -U requests==2.5.3) and see if it helps.

[voronaff]

I tried and it didn't help

File "/usr/local/lib/python2.7/dist-packages/docker/client.py", line 70, in __init__
    self._version = self._retrieve_server_version()
  File "/usr/local/lib/python2.7/dist-packages/docker/client.py", line 90, in _retrieve_server_version
    'Error while fetching server API version: {0}'.format(e)
docker.errors.DockerException: Error while fetching server API version: a float is required

Do you have any thoughts?

[voronaff]

I've found a solution to fix the problem
there is note in requests package

:type timeout: float or tuple

you need to add float in _set_request_timeout method in client.py:

kwargs.setdefault('timeout', float(self.timeout))

[voronaff]

@shin- , what do you think? is it a bug? are you going to fix the bug?

[voronaff]

Sorry guys.
It was my mistake. I use argparse in my script and it returns string instead int. I needed to specify type requested variable.

[shin-]

Glad you were able to figure it out! I'll close this then.