Docker-py: paramiko ssh transport leads to strange discrepancies to docker cli

[riedel]

E.g. when using compose the dependency to paramiko generates avoidable problems (by using the ssh binary in the path). compose and the normal docker-cli work totally different. Furthermore verbose output is generated on stdout.

Paramiko does not obey the .ssh/config as #2251 already pointed out. Far more fatal it does e.g. not support jump hosts or any custom auth schemes.

IMHO it would be the far better choice also removing not so easy to handle second-level dependencies like py-nacl (does not really build via pip in cygwin) to just use the command line ssh in the path as docker-cli does. Paramiko should be e.g. only used when using e.g a --native-ssh switch as i think it is given in docker-machine.

Is there actually any downside in using the systems ssh client?

[riedel]

I quickly fixed most of the stuff that annoyed me here in #2290 by adding limited ssh-config support. Still it makes no sense to me to not rather use the command line version (also from a security point of view) like the docker cli or git or rsync or you name it.

[jayfk]

I second that. I've seen some very strange hangs using SSH transport in docker-compose which uses docker-py under the hood. Running a compose file with SSH transport on macOS takes minutes, whereas on Linux it takes only a couple of seconds.

On top of that, I've seen issues with password protected keys (which are added to ssh-agent) and wrong key usage on macOS.

All of this is very hard to debug and annoying. If you can ssh into the box just fine and the docker cli works, docker-py and dependent libraries should work too.

Implementing all ssh usages people have and handling arcane bugs in paramiko shouldn't be a scope of this project. All of this could be avoided by simply using the default ssh binary.

[zmunro]

@ndeloof this is definitely a bug and not a wanted enhancement. I am unable to run docker-compose with a context that is on a remote VM where I am connected to it using ssh.

[amovfx]

Running into this same problem as well.

[riedel]

One really straight way to fix this quickly btw is use ssh socket forwarding (keeps access control local to user in contrast to port forwarding).

It would probably not cost too much code to open a socket forward before using the standard socket implementation. It might get a bit strange on windows (I think I remember that I had some issues here) , but works with cygwin.

A quick hack is to write a wrapper, that replaces the environment var (also parsing the .env file) before starting the actual compose.

[gabrieldemarmiesse]

Hi all, I made a Python client for Docker that sits on top of the Docker client binary (the one written in go). It took me several months of work. It notably has support for ssh, and is using the ssh connection of the Docker client, so no surprises with paramiko anymore.

It's currently only available for my sponsors, but It'll be open source with an MIT licence May 1st, 2021 🙂

https://gabrieldemarmiesse.github.io/python-on-whales/

[gabrieldemarmiesse]

Hi all, in the end, making Python-on-whales pay-to-use wasn't a success. So I've open-sourced it.

It's free and on Pypi now. Have fun 😃

$ pip install python-on-whales
$ python
>>> from python_on_whales import DockerClient
>>> docker = DockerClient(host="ssh://[email protected]")
>>> docker.ps()
[<python_on_whales.components.container.Container at 0x7fdb6b166b50>,
 <python_on_whales.components.container.Container at 0x7fdb6b166dc0>,
 <python_on_whales.components.container.Container at 0x7fdb6bf15970>,
 <python_on_whales.components.container.Container at 0x7fdb6bf158e0>,
 <python_on_whales.components.container.Container at 0x7fdb6bf158b0>,
 <python_on_whales.components.container.Container at 0x7fdb6bf210a0>,
 <python_on_whales.components.container.Container at 0x7fdb6bf21850>]