Repo2docker: Consider handling docker client init errors

[mukundans91]

I tried to run the command
$ jupyter-repo2docker --no-run --image-name test/test:1.0 https://github.com/jakevdp/PythonDataScienceHandbook
I got a long exception traceback like so

Cloning into '/var/folders/sd/2vfz67290916z1dlgy00jv700000gn/T/repo2dockeryhkrx5g8'...
remote: Counting objects: 1623, done.
remote: Total 1623 (delta 0), reused 0 (delta 0), pack-reused 1623
Receiving objects: 100% (1623/1623), 96.08 MiB | 5.80 MiB/s, done.
Resolving deltas: 100% (1139/1139), done.
Using repo2docker-python3.5 builder
Traceback (most recent call last):
  File "/workspace/py3Env/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/workspace/py3Env/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/workspace/py3Env/lib/python3.6/site-packages/docker/transport/unixconn.py", line 46, in connect
    sock.connect(self.unix_socket)
ConnectionRefusedError: [Errno 61] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspace/py3Env/lib/python3.6/site-packages/requests/adapters.py", line 440, in send
    timeout=timeout
  File "/workspace/py3Env/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/workspace/py3Env/lib/python3.6/site-packages/urllib3/util/retry.py", line 357, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/workspace/py3Env/lib/python3.6/site-packages/urllib3/packages/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/workspace/py3Env/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/workspace/py3Env/lib/python3.6/site-packages/urllib3/connectionpool.py", line 357, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/workspace/py3Env/lib/python3.6/site-packages/docker/transport/unixconn.py", line 46, in connect
    sock.connect(self.unix_socket)
urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionRefusedError(61, 'Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspace/py3Env/lib/python3.6/site-packages/docker/api/client.py", line 168, in _retrieve_server_version
    return self.version(api_version=False)["ApiVersion"]
  File "/workspace/py3Env/lib/python3.6/site-packages/docker/api/daemon.py", line 177, in version
    return self._result(self._get(url), json=True)
  File "/workspace/py3Env/lib/python3.6/site-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/workspace/py3Env/lib/python3.6/site-packages/docker/api/client.py", line 191, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/workspace/py3Env/lib/python3.6/site-packages/requests/sessions.py", line 521, in get
    return self.request('GET', url, **kwargs)
  File "/workspace/py3Env/lib/python3.6/site-packages/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/workspace/py3Env/lib/python3.6/site-packages/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/workspace/py3Env/lib/python3.6/site-packages/requests/adapters.py", line 490, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionRefusedError(61, 'Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspace/py3Env/bin/jupyter-repo2docker", line 11, in <module>
    load_entry_point('jupyter-repo2docker', 'console_scripts', 'jupyter-repo2docker')()
  File "/workspace/py3Env/workspace/repo2docker/repo2docker/__main__.py", line 6, in main
    f.start()
  File "/workspace/py3Env/workspace/repo2docker/repo2docker/app.py", line 394, in start
    for l in picked_buildpack.build(self.output_image_spec, self.build_memory_limit):
  File "/workspace/py3Env/workspace/repo2docker/repo2docker/buildpacks/base.py", line 404, in build
    **docker.utils.kwargs_from_env())
  File "/workspace/py3Env/lib/python3.6/site-packages/docker/api/client.py", line 149, in __init__
    self._version = self._retrieve_server_version()
  File "/workspace/py3Env/lib/python3.6/site-packages/docker/api/client.py", line 176, in _retrieve_server_version
    'Error while fetching server API version: {0}'.format(e)
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', ConnectionRefusedError(61, 'Connection refused'))

Though by parsing through the error callback, it is clearly visible that docker has not been started in my computer, it is not readily apparent that it is an issue with docker during client initialization.

Can this exception be handled by a try/except block during APIClient init and a nicer message printed out while the traceback is printed on the --debug flag being enabled?

[betatim]

We could even check early on when repo2docker is launched whether or not we can successfully contact the docker daemon. This way we won't end up cloning the repo, etc. If the user specified --no-build we should turn off this check?

[mukundans91]

True when --no-build is specified, the check should be turned off.

[choldgraf]

This sounds fine to me - here's where the build logic is checked: https://github.com/jupyter/repo2docker/blob/master/repo2docker/app.py#L506

[yuvipanda]

+1, sounds like a good addition!

[mukundans91]

Is there a way to disable docker for a particular test in travis? That way, we can test if only the log is printed and on debug the stacktrace is printed.

[betatim]

To disable docker we would have to make a new entry in the build matrix I think. A slightly more hacky way is to set DOCKER_HOST to a nonsense value to simulate the lack of docker? There has to be a better way though :-/

[yuvipanda]

Setting DOCKER_HOST to a nonsensical value seems like a great way to test!

On Tue, Jan 9, 2018 at 1:54 PM, Tim Head [e-mail] wrote:

To disable docker we would have to make a new entry in the build matrix I
think. A slightly more hacky way is to set DOCKER_HOST to a nonsense
value to simulate the lack of docker? There has to be a better way though
:-/

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/jupyter/repo2docker/issues/177#issuecomment-356426536,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAB23tppMrcQsoQN0YvwNYlpd44aU_oMks5tI9__gaJpZM4RL5kl
.

--
Yuvi Panda T
http://yuvi.in/blog

[betatim]

One thing we need to be quite sure about is that we restore it to the correct value "no matter what". Or at least leave a big big warning to our future selves explaining why potentially completely weird stuff is happening to tests after this test.

[yuvipanda]

Since we use subprocess to call r2d in most of our tests, we can just set
the environment variable for the one test we need to test it for. That way
it's not travis specific.

On Tue, Jan 9, 2018 at 1:59 PM, Tim Head [e-mail] wrote:

One thing we need to be quite sure about is that we restore it to the
correct value "no matter what". Or at least leave a big big warning to our
future selves explaining why potentially completely weird stuff is
happening to tests after this test.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/jupyter/repo2docker/issues/177#issuecomment-356427919,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAB23vA9oUpTZEY0eWmnQ2Vkh1OdP4BYks5tI-FSgaJpZM4RL5kl
.

--
Yuvi Panda T
http://yuvi.in/blog