Compose: Docker Compose doesn't work when SSH connection used to remote Docker Engine

[raesene]

Description of the issue

Just trying out the new SSH connection introduced in Docker 18.09 and I noticed an error when attempting to do docker-compose up whilst targeting a remote Docker Engine instance.

Errors message below appears to indicate that Compose isn't aware of the SSH protocol for this purpose

docker.errors.DockerException: Invalid bind address protocol: ssh://xfoxy.secinternal.local
[486] Failed to execute script docker-compose

Context information (for bug reports)

Output of docker-compose version

docker-compose version 1.23.1, build b02f1306
docker-py version: 3.5.0
CPython version: 3.6.7
OpenSSL version: OpenSSL 1.1.0f  25 May 2017

Output of docker version

Client:
 Version:           18.09.0
 API version:       1.39
 Go version:        go1.10.4
 Git commit:        4d60db4
 Built:             Wed Nov  7 00:49:01 2018
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.0
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.4
  Git commit:       4d60db4
  Built:            Wed Nov  7 00:16:44 2018
  OS/Arch:          linux/amd64
  Experimental:     false

Output of docker-compose config
(Make sure to add the relevant -f and other flags)

networks:
  testnet: {}
services:
  dradis:
    image: raesene/dradis
    networks:
      testnet: null
    ports:
    - 3000/tcp
    volumes:
    - data:/data:rw
  sectest:
    image: raesene/sectest
    networks:
      testnet: null
    ports:
    - 22/tcp
    volumes:
    - data:/data:rw
version: '3.0'
volumes:
  data: {}

Steps to reproduce the issue

1. Configure a Docker client (18.09) to connect to a remote Docker engine instance via SSH
2. Run docker-compose up in a directory with a docker-compose.yml file.
3. Error occors.

Observed result

Error occurs

Expected result

Docker compose contacts the remote docker engine instance to create the containers.

Stacktrace / full error message

Traceback (most recent call last):
  File "bin/docker-compose", line 6, in <module>
  File "compose/cli/main.py", line 71, in main
  File "compose/cli/main.py", line 124, in perform_command
  File "compose/cli/command.py", line 42, in project_from_options
  File "compose/cli/command.py", line 123, in get_project
  File "compose/cli/command.py", line 94, in get_client
  File "compose/cli/docker_client.py", line 127, in docker_client
  File "site-packages/docker/api/client.py", line 118, in __init__
  File "site-packages/docker/utils/utils.py", line 256, in parse_host
docker.errors.DockerException: Invalid bind address protocol: ssh://xfoxy.secinternal.local
[486] Failed to execute script docker-compose

Additional information

Client is WSL (Ubuntu 18.04) Server is Ubuntu 18.04 running Docker 18.09.

[shin-]

Support for the SSH protocol will be added in the next version of Compose. https://github.com/docker/docker-py/issues/2159

[raesene]

Cool Thanks for the info. :)

[mmiglier]

Tested this with docker-compose version 1.24.0-rc1 but it doesn't look like it is reading my ~/.ssh/config. DOCKER_HOST=ssh://my.docker.host docker ps works fine while DOCKER_HOST=ssh://my.docker.host docker-compose ps gives me paramiko.ssh_exception.AuthenticationException: Authentication failed.

[y0zg]

Tested this with docker-compose version 1.24.0-rc1 but it doesn't look like it is reading my ~/.ssh/config. DOCKER_HOST=ssh://my.docker.host docker ps works fine while DOCKER_HOST=ssh://my.docker.host docker-compose ps gives me paramiko.ssh_exception.AuthenticationException: Authentication failed.

settings sshd config to MaxSessions 30 helped as workaround

[tskipa]

Any update on this?

[ssbarnea]

Apparently switching to edge version on mac (which uses 1.24) makes it work but is really inconvenient as edge comes with its own set of issues.

[fzamperin]

Confirm that using docker-compose -H ssh://[email protected] up gives me:

/var/folders/vn/5g8_6g0s63s0zzz8kcdnxx2c0000gn/T/_MEIub7S75/paramiko/ecdsakey.py:164: CryptographyDeprecationWarning: Support for unsafe construction of public numbers from encoded data will be removed in a future version. Please use EllipticCurvePublicKey.from_encoded_point
Traceback (most recent call last):
File "docker-compose", line 6, in
File "compose/cli/main.py", line 71, in main
File "compose/cli/main.py", line 124, in perform_command
File "compose/cli/command.py", line 42, in project_from_options
File "compose/cli/command.py", line 123, in get_project
File "compose/cli/command.py", line 94, in get_client
File "compose/cli/docker_client.py", line 127, in docker_client
File "site-packages/docker/api/client.py", line 166, in __init__
File "site-packages/docker/transport/sshconn.py", line 84, in __init__
File "site-packages/docker/transport/sshconn.py", line 94, in _connect
File "site-packages/paramiko/client.py", line 343, in connect
File "site-packages/paramiko/util.py", line 280, in retry_on_signal
File "site-packages/paramiko/client.py", line 343, in
TimeoutError: [Errno 60] Operation timed out
[21295] Failed to execute script docker-compose

But with docker command docker -H ssh://[email protected] ps works normally, using docker-compose version 1.24.1, build 4667896b on Mac OS and Docker version 19.03.1, build 74b1e89

[cuttingd]

Any news on this? I'm running this: docker-compose -H ssh://user@host down and I get the following:

Traceback (most recent call last):
File "docker-compose", line 6, in
File "compose\climain.py", line 71, in main
File "compose\climain.py", line 124, in perform_command
File "compose\cli\command.py", line 42, in project_from_options
File "compose\cli\command.py", line 123, in get_project
File "compose\cli\command.py", line 94, in get_client
File "compose\cli\docker_client.py", line 127, in docker_client
File "site-packages\docker\api\client.py", line 166, in __init__
File "site-packages\docker\transport\sshconn.py", line 84, in __init__
File "site-packages\docker\transport\sshconn.py", line 94, in _connect
File "site-packages\paramiko\client.py", line 437, in connect
File "site-packages\paramiko\client.py", line 676, in _auth
File "site-packages\paramiko\agent.py", line 379, in __init__
File "site-packages\paramiko\agent.py", line 65, in _connect
File "site-packages\paramiko\agent.py", line 83, in _send_message
File "site-packages\paramiko\agent.py", line 91, in _read_all
paramiko.ssh_exception.SSHException: lost ssh-agent
[16740] Failed to execute script docker-compose

[spirit986]

Can confirm I am having the same issue as @raesene

[jayfk]

I believe that most of the SSH related problems stem from the fact that docker-py (which is used by docker-compose) uses the paramiko ssh library instead of defaulting to the ssh binary like the docker cli.

There's an issue in docker-py discussing this: https://github.com/docker/docker-py/issues/2289

[dockermaestro]

Any update?

[zmunro]

Why is this issue closed? I am still unable to set a docker-compose context to a docker engine running on a remote server over ssh.

[bvinayprasad]

I'm seeing this error when running docker-compose with -H option
docker.errors.DockerException: Invalid bind address protocol: ssh://[email protected]

[amovfx]

Same problems here

[mmoya]

In the meantime, this workaround works for me:

1. Launch socat in a terminal:

   ssh -t -L 8888:127.0.0.1:8888 remote-docker-host 'socat TCP4-LISTEN:8888,fork,bind=127.0.0.1 UNIX-CONNECT:/var/run/docker.sock'

2. Point compose to local port 8888

   docker-compose -H tcp://:8888 up

[cde0]

Why this issue is close i don't think is fix.

[KatSick]

Yep. Looks like docker-compose does not use ~/.ssh/config when using DOCKER_HOST variable, while docker does

[KatSick]

This workaround helps https://github.com/docker/compose/issues/6513#issuecomment-613358317

[hazcod]

Seeing this problem as wel with a super simple deploy over SSH.

[eliyahus]

Same issue. docker commands work fine with the SSH connections, while docker-compose commands don't.

Traceback (most recent call last):
File "docker-compose", line 6, in
File "compose\climain.py", line 72, in main
File "compose\climain.py", line 125, in perform_command
File "compose\cli\command.py", line 76, in project_from_options
File "compose\cli\command.py", line 142, in get_project
File "compose\clidocker_client.py", line 47, in get_client
File "compose\clidocker_client.py", line 174, in docker_client
File "site-packagesdocker\api\client.py", line 166, in __init__
File "site-packagesdocker\transportsshconn.py", line 111, in __init__
File "site-packagesdocker\transportsshconn.py", line 119, in _connect
File "site-packagesparamiko\client.py", line 446, in connect
File "site-packagesparamiko\client.py", line 765, in _auth
paramiko.ssh_exception.SSHException: No authentication methods available
[14836] Failed to execute script docker-compose

Update: eventually, I solved this one using this tip: https://github.com/docker/compose/issues/6655#issuecomment-541394093

[hazcod]

Interestingly things work fine with docker stack deploy -c compose.yml since that doesn't use the Python library, altough that requires a Swarm or Kubernetes host.

[snwfdhmp]

I'm having the same issue, cannot deploy over ssh

[mmenapace]

same problem here.
On local context everything works like a charm.
When I try to launch the docker-compose on a remote host via ssh the docker engine on the stops and restarts the services but doesn't update it.
command "trace"

• created context "remote" with docker endpoint ssh://[user]@[local ip]
• docker context use remote
• [rebuild solution] (gradle clean build) -> docker file for bootJar copy
• docker-compose up
  The service on remote starts but with an old version of the solution

[julianwagle]

still an issue:
(base) home@J django-on-docker % DOCKER_HOST=ssh://[email protected] docker-compose -f docker-compose.staging.yml up -d --build
Traceback (most recent call last):
File "docker-compose", line 6, in
File "compose/cli/main.py", line 72, in main
File "compose/cli/main.py", line 125, in perform_command
File "compose/cli/command.py", line 76, in project_from_options
File "compose/cli/command.py", line 142, in get_project
File "compose/cli/docker_client.py", line 47, in get_client
File "compose/cli/docker_client.py", line 174, in docker_client
File "site-packages/docker/api/client.py", line 166, in __init__
File "site-packages/docker/transport/sshconn.py", line 111, in __init__
File "site-packages/docker/transport/sshconn.py", line 119, in _connect
File "site-packages/paramiko/client.py", line 349, in connect
File "site-packages/paramiko/util.py", line 283, in retry_on_signal
File "site-packages/paramiko/client.py", line 349, in
TimeoutError: [Errno 60] Operation timed out
[61197] Failed to execute script docker-compose