Machine: scp doesn't work on redhat6 and flavors (centos6,oraclelinux6)

[JonathanHuot]

I am not able to use docker-machine scp because it is automatically adding -3 option which is not available on some distributions.

Distributions tested are redhat6-like, centos6, oraclelinux6.
Reproductions steps:
_Centos 6_

$ cat Dockerfile
FROM centos:6

RUN yum install -y openssh-clients

RUN touch empty
RUN scp -3 empty empty3
$ docker build -t scp-3 .
(..)
scp: illegal option -- 3
usage: scp [-1246BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file]
           [-l limit] [-o ssh_option] [-P port] [-S program]
           [[user@]host1:]file1 ... [[user@]host2:]file2
The command '/bin/sh -c scp -3 empty empty3' returned a non-zero code: 1

_OracleLinux 6_

$ cat Dockerfile
FROM oraclelinux:6

RUN touch empty
RUN scp -3 empty empty3
$ docker build -t scp-3 .
(..)
scp: illegal option -- 3
usage: scp [-1246BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file]
           [-l limit] [-o ssh_option] [-P port] [-S program]
           [[user@]host1:]file1 ... [[user@]host2:]file2
The command '/bin/sh -c scp -3 empty empty3' returned a non-zero code: 1

That's would be great if -3 option can be optional or if we can override the defaults.

[bamarni]

It seems like something internal which shouldn't be exposed imo. How about setting this option only if both source and destination are remote machines?

[JonathanHuot]

In the other hand, we could consider docker-machine scp as a pass through of the scp command, and consider adding options automatically is bad because it could be not supported on the docker-machine host.

For information, I forgot to paste the error we got when running docker-machine scp on any redhat6 flavors:

$ docker-machine scp x x
scp: illegal option -- 3
usage: scp [-1246BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file]
           [-l limit] [-o ssh_option] [-P port] [-S program]
           [[user@]host1:]file1 ... [[user@]host2:]file2
exit status 1

[JonathanHuot]

Note that if others are in the same situation, we can hack scp command to remove invalid flags, as a workaround:

Creating the file (chmod +x)

$ cat ~/bin/scp
#!/bin/bash
$(eval echo /usr/bin/scp $*|sed s/-3//)

Testing with docker-machine

$ docker-machine scp empty z
scp: illegal option -- 3
usage: scp [-1246BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file]
           [-l limit] [-o ssh_option] [-P port] [-S program]
           [[user@]host1:]file1 ... [[user@]host2:]file2
exit status 1
$ export PATH=~/bin:$PATH
$ docker-machine scp empty z
$ 

[bamarni]

Making it explicit is a solution indeed, I was just trying to find something which doesn't break compatibility.

If possible it would be even better to detect if the option is available, it was apparently already a concern when it was added (cf. https://github.com/docker/machine/blob/5ce0ab2d12aec5b9128d6ac6575c04e73552029d/commands/scp.go#L93).

[afbjorklund]

Seems to have been added in https://github.com/openssh/openssh-portable/commit/f12114366b4ffcd34e3a638dd187f29ac03fbdbd (5.7p1)

So it's not available in CentOS-6 (5.3p1) but only in CentOS-7 (6.6.1p1)
But an opt-out boolean option would maybe be a possible solution to this.

However, it would probably be better to do _two_ copies in that legacy case ?

1. Copy from remote # 1 to localhost 2. Copy from localhost to remote # 2

Otherwise it would try to get _directly_ from remote # 1 to remote # 2:

"Without this option the data is copied directly between the two remote hosts."

[austinrr]

Just ran into this issue using Docker for Windows while trying to follow the tutorial here: https://docs.docker.com/engine/getstarted-voting-app/deploy-app/

Documents $ docker-machine scp localhost:/c/Users/austi/Documents/docker-stack.yml manager: scp.exe": illegal option -- 3 usage: scp [-1246BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file] [-l limit] [-o ssh_option] [-P port] [-S program] [[user@]host1:]file1 ... [[user@]host2:]file2 exit status 1

````
Documents $ docker version
time="2017-03-23T16:55:24-04:00" level=info msg="Unable to use system certificate pool: crypto/x509: system root pool is not available on Windows"
Client:
Version: 17.03.0-ce
API version: 1.26
Go version: go1.7.5
Git commit: 60ccb22
Built: Thu Feb 23 10:40:59 2017
OS/Arch: windows/amd64

Server:
Version: 17.03.0-ce
API version: 1.26 (minimum version 1.12)
Go version: go1.7.5
Git commit: 3a232c8
Built: Tue Feb 28 07:52:04 2017
OS/Arch: linux/amd64
Experimental: false
````

[afbjorklund]

As a workaround, you can use the new --delta option from https://github.com/docker/machine/pull/4019
Since rsync doesn't do remote-to-remote, it will use localhost (like -3)

But it would probably be a good idea to rewrite scp without using -3,
like just using a "mktemp" directory if both arguments are remote ?

[afbjorklund]

@JonathanHuot : note that dropping the -3 will try to do remote-to-remote:

 -3      Copies between two remote hosts are transferred through the local host.
         Without this option the data is copied directly between the two remote hosts.

So it needs rewriting (or a wrapper), with a temporary local dir (and files)...