Compose: All docker-compose (but not docker) commands time out: "An http request took too long to complete."

[su-narthur]

I'm getting the following error when running docker-compose commands:

ERROR: An HTTP request took too long to complete. Retry with --verbose to obtain debug information. If you encounter this issue regularly because of slow network conditions, consider setting COMPOSE_HTTP_TIMEOUT to a higher value (current value: 60).

Once I start getting the errors, I consistently get the errors for docker-compose commands until I restart Docker via the toolbar menu.

• Turning off wifi doesn't help
• Adding these entries to /etc/hosts doesn't help:

127.0.0.1       localunixsocket.local  
127.0.0.1       localunixsocket  
127.0.0.1       localunixsocket.lan

• Docker diagnose comes up clean
• Stopping all other containers than the 3 needed for this app doesn't help. All remaining containers are healthy.
• ping localhost works
• docker container ls (and other docker commands) works instantly
• I can successfully telnet into all my running containers' published ports
• I can use my app via the browser without any problems
• No docker updates available ("Docker 17.12.0-ce-mac49 is currently the newest version available.")
• I have no reference to tty in my docker-compose.yml or Dockerfile files

OS: macOS High Sierra version 10.13.2 (17C205)
Processor: 3.4 GHz Intel Core i5
Memory: 40 GB 2400 MHz DDR4 (I'm not using anywhere near that much)

My docker diagnostics ID is 0F286399-29FA-49AB-A3E7-669DB39AD08B

Docker for Mac: version: 17.12.0-ce-mac49 (d1778b704353fa5b79142a2055a2c11c8b48a653)
macOS: version 10.13.2 (build: 17C205)
logs: /tmp/0F286399-29FA-49AB-A3E7-669DB39AD08B/20180128-105656.tar.gz
[OK] db.git
[OK] vmnetd
[OK] dns
[OK] driver.amd64-linux
[OK] virtualization VT-X
[OK] app
[OK] moby
[OK] system
[OK] moby-syslog
[OK] kubernetes
[OK] env
[OK] virtualization kern.hv_support
[OK] slirp
[OK] osxfs
[OK] moby-console
[OK] logs
[OK] docker-cli
[OK] menubar
[OK] disk

[shin-]

Retry with --verbose to obtain debug information.

Did you do this? If so, can you share the output?

[su-narthur]

@shin- Same as when there isn't an issue and you add the --verbose argument--it gives you the command's usage since docker-compose commands don't seem to support that argument.

[shin-]

It needs to be before the subcommand (ie docker-compose --verbose up instead of docker-compose up --verbose)

[su-narthur]

@shin- Oooh! Yeah, that would make a difference. I'll try this next time I have the issue.

[su-narthur]

@shin- Ok, it happened again, and here is the verbose version:

$ docker-compose --verbose logs
compose.config.config.find: Using configuration files: ./docker-compose.yml
docker.auth.find_config_file: Trying paths: ['/Users/narthur/.docker/config.json', '/Users/narthur/.dockercfg']
docker.auth.find_config_file: Found file at path: /Users/narthur/.docker/config.json
docker.auth.load_config: Couldn't find 'auths' or 'HttpHeaders' sections
docker.auth.parse_auth: Auth data for {0} is absent. Client might be using a credentials store instead.
urllib3.connectionpool._make_request: http://localhost:None "GET /v1.25/version HTTP/1.1" 200 566
compose.cli.command.get_client: docker-compose version 1.18.0, build 8dd22a9
docker-py version: 2.6.1
CPython version: 2.7.12
OpenSSL version: OpenSSL 1.0.2j  26 Sep 2016
compose.cli.command.get_client: Docker base_url: http+docker://localunixsocket
compose.cli.command.get_client: Docker version: KernelVersion=4.9.60-linuxkit-aufs, Components=[{u'Version': u'17.12.0-ce', u'Name': u'Engine', u'Details': {u'KernelVersion': u'4.9.60-linuxkit-aufs', u'Os': u'linux', u'BuildTime': u'2017-12-27T20:12:29.000000000+00:00', u'ApiVersion': u'1.35', u'MinAPIVersion': u'1.12', u'GitCommit': u'c97c6d6', u'Arch': u'amd64', u'Experimental': u'true', u'GoVersion': u'go1.9.2'}}], Arch=amd64, BuildTime=2017-12-27T20:12:29.000000000+00:00, ApiVersion=1.35, Platform={u'Name': u''}, Version=17.12.0-ce, MinAPIVersion=1.12, GitCommit=c97c6d6, Os=linux, Experimental=True, GoVersion=go1.9.2
compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={u'label': [u'com.docker.compose.project=sudocker', u'com.docker.compose.oneoff=False']})
urllib3.connectionpool._make_request: http://localhost:None "GET /v1.25/containers/json?all=1&limit=-1&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Dsudocker%22%2C+%22com.docker.compose.oneoff%3DFalse%22%5D%7D&trunc_cmd=0&size=0 HTTP/1.1" 200 4338
compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 3 items)
compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- (u'6c927f8d3ce359352f193a0d85b7bafc8d9a2ab3f83fd590e2617c6c010a65bd')
ERROR: compose.cli.errors.log_timeout_error: An HTTP request took too long to complete. Retry with --verbose to obtain debug information.
If you encounter this issue regularly because of slow network conditions, consider setting COMPOSE_HTTP_TIMEOUT to a higher value (current value: 60).

[shin-]

@su-narthur Thanks! If you run docker container inspect 6c927f8d3ce359352f193a0d85b7bafc8d9a2ab3f83fd590e2617c6c010a65bd, do you see any lag / slowdown or error show up?

[su-narthur]

@shin- I waited for 13.5 minutes and it was just hanging. No output at all.

[su-narthur]

I ended up manually ctrl+c canceling it.

[shin-]

@su-narthur Sounds like this is an issue with Docker For Mac in this case. Do you mind creating an issue on the docker/for-mac repo with the information you provided here? It might be worth running another diagnostic with the latest info as well.

[su-narthur]

@shin- Yup, there was already an issue there. I added my info to it.

https://github.com/docker/for-mac/issues/2378

Thanks for your help!

[Routhinator]

@su-narthur @shin- I think this was closed too hastily. I've been getting this off and on since January of this year. But not on Mac, on Ubuntu. First on 16.04, then on 17.04, 17.10 and now on 18.04 - and my verbose output hangs at the exact same spot as @su-narthur 's, during container inspection.

docker-compose --verbose run playbook -i inventory/global/ playbooks/dev-jasper-reports.yml
compose.config.config.find: Using configuration files: ./docker-compose.yml
docker.utils.config.find_config_file: Trying paths: ['/home/crouth/.docker/config.json', '/home/crouth/.dockercfg']
docker.utils.config.find_config_file: Found file at path: /home/crouth/.docker/config.json
docker.auth.load_config: Found 'auths' section
urllib3.connectionpool._make_request: http://localhost:None "GET /v1.24/version HTTP/1.1" 200 543
compose.cli.command.get_client: docker-compose version 1.21.2, build a133471
docker-py version: 3.3.0
CPython version: 3.6.5
OpenSSL version: OpenSSL 1.0.1t  3 May 2016
compose.cli.command.get_client: Docker base_url: http+docker://localhost
compose.cli.command.get_client: Docker version: Platform={'Name': ''}, Components=[{'Name': 'Engine', 'Version': '18.06.0-ce', 'Details': {'ApiVersion': '1.38', 'Arch': 'amd64', 'BuildTime': '2018-07-18T19:07:56.000000000+00:00', 'Experimental': 'false', 'GitCommit': '0ffa825', 'GoVersion': 'go1.10.3', 'KernelVersion': '4.15.0-30-generic', 'MinAPIVersion': '1.12', 'Os': 'linux'}}], Version=18.06.0-ce, ApiVersion=1.38, MinAPIVersion=1.12, GitCommit=0ffa825, GoVersion=go1.10.3, Os=linux, Arch=amd64, KernelVersion=4.15.0-30-generic, BuildTime=2018-07-18T19:07:56.000000000+00:00
compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('jasperreportsservers_default')
urllib3.connectionpool._make_request: http://localhost:None "GET /v1.24/networks/jasperreportsservers_default HTTP/1.1" 404 61
compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('jasperreports-servers_default')
urllib3.connectionpool._make_request: http://localhost:None "GET /v1.24/networks/jasperreports-servers_default HTTP/1.1" 200 862
compose.cli.verbose_proxy.proxy_callable: docker inspect_network -> {'Attachable': True,
 'ConfigFrom': {'Network': ''},
 'ConfigOnly': False,
 'Containers': {'35ef0763b3a38a776409ef8d426f9fd1eb71549cb13c4166c638d33b7c77bd34': {'EndpointID': '46aaa4306520341b9d1f3f238ceebca2d1aa9f9b00bf9c2b221fa6313439ebc7',
                                                                                     'IPv4Address': '172.18.0.2/16',
                                                                                     'IPv6Address': '',
                                                                                     'MacAddress': '02:42:ac:12:00:02',
                                                                                     'Name': 'jasperreports-servers_playbook_run_112'}},
 'Created': '2018-06-27T13:27:42.910819198-07:00',
 'Driver': 'bridge',
...
compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('dockerhub.alea.ca/docker/ansible')
urllib3.connectionpool._make_request: http://localhost:None "GET /v1.24/images/dockerhub.alea.ca/docker/ansible/json HTTP/1.1" 200 None
compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64',
 'Author': '',
 'Comment': '',
 'Config': {'ArgsEscaped': True,
            'AttachStderr': False,
            'AttachStdin': False,
            'AttachStdout': False,
            'Cmd': None,
            'Domainname': '',
            'Entrypoint': ['ansible-playbook'],
...
compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=jasperreports-servers', 'com.docker.compose.service=playbook', 'com.docker.compose.oneoff=True']})
urllib3.connectionpool._make_request: http://localhost:None "GET /v1.24/containers/json?limit=-1&all=1&size=0&trunc_cmd=0&filters=%7B%22label%22%3A+%5B%22com.docker.compose.project%3Djasperreports-servers%22%2C+%22com.docker.compose.service%3Dplaybook%22%2C+%22com.docker.compose.oneoff%3DTrue%22%5D%7D HTTP/1.1" 200 None
compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 112 items)
compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('35ef0763b3a38a776409ef8d426f9fd1eb71549cb13c4166c638d33b7c77bd34')
ERROR: compose.cli.errors.log_timeout_error: An HTTP request took too long to complete. Retry with --verbose to obtain debug information.
If you encounter this issue regularly because of slow network conditions, consider setting COMPOSE_HTTP_TIMEOUT to a higher value (current value: 60).

[su-narthur]

@Routhinator Sounds fair to me, reopening until we hear more from project maintainers.

[alvery]

I can confirm this behaviour, I'm experiencing same on my linode server after reboot. But approx. after three minutes it has started to respond. Guess it could be a problem with docker service start

[shin-]

Hey folks, sorry for the late follow-up.
Given that the docker container inspect command hangs identically, all signs point to this being an engine issue. I see that a report has been made here https://github.com/docker/for-linux/issues/397 (thanks @Routhinator ), which people should follow and contribute to in order to see it resolved.

I'm going to close this again as this is not a Compose issue.

[abdrmdn]

Can you show you're docker-compose.yaml file?
i just had the same issues, and it turned out, it was because one of the images, cannot be pulled.

[lud]

Hello,

I just ran into this problem with the hello-world image. I had another docker-compose up command running in another terminal, in another directory.

But the service that would not start had the same name than a service which was already runnig in the other terminal.

Edit: changed the name and it works, tried to run a mariadb image instead, keeping the new name : it hangs. Now running hello-world with this new name : hangs. Changin the name again : hello-world starts ...

[ggl-tommy-fritz]

I ran into a very similar issue. It would just time out on any type of docker-compose command.

I increased the memory from 2gb to 7gb in the docker for mac settings, and that seemed to fix it for me. YMMV, but I figured it was worth posting this info as I've been having this docker issue for months.

[Anushkasharma]

In case anyone else faces this : restarting docker for mac resolved this for me.

[su-narthur]

@Anushkasharma Yes, restarting Docker for Mac does resolve it temporarily, but that can really slow you down if you have containers that take a while to start up.

[taqvi]

By simply restarting the docker service via sudo service docker restart I was able to get the aforementioned error to go away when running docker-compose up.

Note: I did not touch the tty=true parameter as suggested, as I wanted to avoid a quick fix.

[gabrieldylanls]

I was having the same problem, I just restart my computer and it was resolved.

[aconital]

We are having the same issue on our Ubuntu server. Docker-compose up and down both timeout and docker inspect hangs forever. We have to restart the whole server to resolve the issue.

[aashish108]

I also have the same issue but on my Raspbery Pi 4. Hopefully the 4GB RAM and the CPU are fast enough for small docker-compose? Restarting usually fixes it but it comes back quickly.

[DMW007]

Dont think this is a RPI issue, especially when you have 4GB of memory. On my RPI3 with only 1GB running Postgres and Python scripts, it works well and there is still some memory free. So it shouldn't be an issue, expect you have large applications running which allocates those memory.

I have this issue on a i7 920 with 32GB of memory, where only 19GB are in use. Restarting the Docker daeom (or the complete machine) helps, but it's not a long term solution. After some time, it came back.

It seems an issue of Docker itself, not compose. In the linked Docker issue are people having this problem on plain Docker without Compose.

[jg-26]

I had the same problem and restarting the docker worked. Thank you @taqvi !!