Deployer: Deprecate phpseclib, ext-ssh2 🦋

[antonmedv]

| Q | A
| ----------------- | ---
| Issue Type | Feature Request
| Deployer Version | 5.x

Now Deployer have 3 different implementations of ssh client: phpseclib, ext-ssh2 and native.
I suggest deprecate first two and leave only native implementation with ssh multiplexing turned on by default. Reasons for then is what phpseclib and ext-ssh2 comes with different features and some may work in one, but not in another (like agent forwarding, ssh config, etc.). Also we need to support this three times of ssh libs. It will really simpler in there will be only one: native one. As it will come with all functionality. Also with ssh multiplexing it comes with same speed as phpseclib and ext-ssh2.

[antonmedv]

Deploy time of an laravel application:

| SSH client | Deploy time |
|---|---|
| phpseclib | 1m21.443s |
| ext-ssh2 | 1m16.211s |
| native | 0m53.159s |

[antonmedv]

Plus there will be no need of herzult/php-ssh and phpseclib/phpseclib dependencies.

[antonmedv]

So, i thinking to do it in 2 steps:
First, show deprecation message to all user in 4.2.0:

Warning: ssh type `phpseclib` will be deprecated in Deployer 5.
Add this lines to your deploy.php file:

    set('ssh_type', 'native');
    set('ssh_multiplexing', true);

Second, remove completely phpseclib and ext-ssh2 in Deployer 5. Turn ssh multiplexing by default.

[mbrodala]

Notice that we need at least username/password authentication implemented for native SSH to the replace the other two.

Of course, key-based authentication is preferred, but I can imagine e.g. a task which uploads your public key (and others) to a remote server. For this you need to authenticate via password at least once.

[houstona]

Ran into an issue while trying to use native. I get the POSIX complaint message.

[antonmedv]

@houstona try to run this command ssh user@host "echo $0" in terminal.

[houstona]

I get the following error

ssh [email protected] "echo $0"
Host key verification failed.

Just to make sure you are aware. I am running this through GitLab CI using Docker.

[antonmedv]

You need to sure what when you login to server it uses bash.

[edbizarro]

same problem as @houstona

my server is running zsh + presto

[antonmedv]

@houstona you need to get -bash response from this command.
@edbizarro same for you. What is result of this command?

[dangreaves]

When using native SSH (multiplexing on), I also get the POSIX compliance message and a big red fail. I'm running from Codeship CI.

Has been working fine with phpseclib.

$ ssh user@server "echo $0"
/bin/bash

[antonmedv]

@dangreaves what is your ci config?

[dangreaves]

My Codeship process runs as follows:

# We support all major PHP versions. Please see our docs for a full list
# https://documentation.codeship.com/classic/languages-frameworks/php/
phpenv local 7.1

# Install dependencies through Composer
composer install --prefer-dist --no-interaction

# Upgrade global phpunit
composer global remove phpunit/phpunit
composer global require phpunit/phpunit:~5.7

# Run phpunit
phpunit

# Run deploy
./vendor/bin/dep deploy production

My deploy.php is as follows:

<?php

namespace Deployer;

require 'recipe/laravel.php';

// Configuration

set('repository', /*redacted*/);

add('shared_files', []);
add('shared_dirs', []);

add('writable_dirs', []);

// Servers

server('production', /*redacted*/)
    ->user(/*redacted*/)
    ->identityFile()
    ->set('deploy_path', '/home/user/deployments');

[edbizarro]

@elfet

the result:

▶ ssh staging-core "echo $0"
-zsh

[antonmedv]

@edbizarro zsh isn't supported by Deployer. You need to switch to using bash on shell login.

[houstona]

@elfet I can deploy fine to the server locally but when deploying from a GitLab CI it fails with the POSIX compliant message. When running just "echo $0" from the GitLab CI I get the following

$ echo $0
/bin/bash

[lbausch]

Same here. Same setup (GitLab CI with Docker), works fine when executing manually or using phpseclib, but fails when using "native" with the POSIX compliant message. Server uses bash.

[dangreaves]

To add to this, I just checked my remote server shell from within a Codeship CI build.

rof@railsonfire_666589f4-a6e5-409f-8fce-d969a319712b_f18a3ca16671:~$ ssh user@... "echo $0"
Warning: Permanently added '...' (RSA) to the list of known hosts.
-bash

It's definitely getting -bash back but I still get the POSIX message when multiplexing is switched on with the native SSH type. Without multiplexing enabled, the connections seem to work but run _extremely_ slowly.

[antonmedv]

Ok, lets me check this.

[msbit]

How does one allow pemFile login using native?

EDIT:

NM, can get the same outcome using:

server(...)
  ->identityFile(null, '<path.pem>', null)

[antonmedv]

@msbit use pemFile.

[msbit]

@elfet I might be wrong, but it looks like Deployer\Server\Configuration::getPemFile is only called from the Deployer\Server\PhpSecLib, so if I use native as the ssh_type it won't get called? That matches what I was seeing when testing it.

[antonmedv]

Yes, you are right. This is possible with phpseclib only. I'm going to implement pem auth for native ssh for new Deployer 5.

[houstona]

@elfet Is #1011 the fix for CI builds?

[houstona]

@elfet actually I misread that. Looks like that is just a feature request and not a bug fix.

[antonmedv]

@houstona sorry, still have not time to check this ci bug. A lot of work now. Will try to fix it on next week.

[lasselehtinen]

This error happens with cmder with the packaged git bash. The shell is indeed bash on the server.

[root@someserver~]# echo $0
-bash

$ dep deploy production
➤ Executing task deploy:prepare
      1 [main] ssh 14188 child_info_fork::abort: C:\cmder\vendor\git-for-windows\usr\bin\msys-crypto-1.0.0.dll: Loaded to different address: parent(0xE60000) != child(0xED0000)
control_persist_detach: fork: Resource temporarily unavailable
      1 [main] ssh 2960 child_info_fork::abort: C:\cmder\vendor\git-for-windows\usr\bin\msys-crypto-1.0.0.dll: Loaded to different address: parent(0xFB0000) != child(0xF40000)

  Shell on your server is not POSIX-compliant. Please change to sh, bash or similar.
  Usually, you can change your shell to bash by running: chsh -s /bin/bash
✔                                                                                     

➤ Executing task deploy:failed


  [Symfony\Component\Process\Exception\ProcessFailedException]
  The command "ssh -A -q -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o ControlMaster=auto -o Control
  Persist=5 -o ControlPath='~/.ssh/[email protected]:22' -p "22" -i "K:\/.ssh/id_rsa" "[email protected]" "echo $0"" failed.
  Exit Code: 255(Unknown error)
  Working directory: C:\Users\lasleh\Dropbox\Development\partner-portal
  Output:
  ================
  Error Output:
  ================
        1 [main] ssh 2960 child_info_fork::abort: C:\cmder\vendor\git-for-windows\usr\bin\msys-crypto-1.0.0.dll: Load
  ed to different address: parent(0xFB0000) != child(0xF40000)


deploy [-p|--parallel]

Interesting thing is what when I set the multiplexing to false, I get this:

$ dep deploy production                                                                                                 
➤ Executing task deploy:prepare                                                                                         
✔ Executing task deploy:failed                                                                                          

  [RuntimeException]                                                                                                    
  There already is a directory (not symlink) named "current" in /var/www/partner-portal. Remove this directory so it  can be replaced with a symlink for atomic deployments.                                                                

deploy [-p|--parallel]         

[fnagel]

Works for me (as I only use Deployer within a Linux VM) but what about Windows user?

[antonmedv]

I think in windows broken php func escapeshellarg on Windows. But i found solution for it, will ship it in next release.

[leo108]

Is it possible for NativeSSH to use a RASKey with passphrase without expect support?

[antonmedv]

1092

[lasselehtinen]

The latest version (4.3.0) seems to fix my issue, but it seems like the multiplexing is broken on Cygwin / Git Bash on Windows.

[RuntimeException]
  The command "ssh -A -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o ControlMaster=auto -o ControlPersist=5 -o ControlPath='~/.ssh/[email protected]:22' -p "22" -i "K:\/.ssh/id_rsa" "[email protected]" bash -s" failed.

  Exit Code: 255(Unknown error)
  Working directory: C:\foobar
  Output:
  ================
  Error Output:
  ================
  Warning: Permanently added 'server.com,1.2.3.4' (ECDSA) to the list of known hosts.
  mux_client_request_session: read from master failed: Connection reset by peer
  Failed to connect to new control master

The SSH with -vvv ends with:

debug1: channel 0: new [/k/.ssh/[email protected]:22]
debug3: muxserver_listen: mux listener channel 0 fd 4
debug2: fd 3 setting TCP_NODELAY
debug3: ssh_packet_set_tos: set IP_TOS 0x08
debug1: control_persist_detach: backgrounding master process
debug2: control_persist_detach: background process is 12204
debug1: forking to background
debug2: fd 4 setting O_NONBLOCK
buffer_append_space: insufficient buffer space

I can find lot questions about the same problem in different context, but at least I could not find any definite solution for this. Setting the ssh_multiplexing to false fixes this issue. So this has nothing to do with deployer, I wrote just in case somebody else bumps to the same problem.

[antonmedv]

@lasselehtinen I've made workaround on this problem in next. Where will be option git_tty which allow to use git with multiplexing on. And also allow to write password for git keys if needed.

[murilodbva]

Hi everybody. I upgraded from version 4.0.2 to 4.3.0.
After that it show the message for add these two lines:
set('ssh_type', 'native');
set('ssh_multiplexing', true);

I did.
Now the deploy works, but the deployer is asking the server's password all the time when I do dep deploy name-server.
I can't understand why the deployer is ignoring my server password configured in deploy.php.

The contain of my deploy.php:

namespace Deployer;
require_once  'recipe/common.php';

set('ssh_type', 'native');
set('ssh_multiplexing', true);

set('use_ssh2', false);
set('projectName','xyz');
set('shared_dirs', ['files', 'public']);
set('writable_dirs', ['files', 'public']);

server('homolog', 'adress.com.br', 99)
    ->user('user')
    ->password('xxxxxx')
    ->set('deploy_path','/var/www/{{projectName}}')
    ->stage('homologacao');

set('repository', '[email protected]:fulano/xyz.git');

task('deploy:shared_rename', function () {
    run("cp {{release_path}}/config.inc.php.dist {{deploy_path}}/shared/config.inc.php && ln -nfs {{deploy_path}}/shared/config.inc.php {{release_path}}/config.inc.php");
    })->desc('Copia configurações ojs');
//
task('deploy:preparando', function () {
      run("sudo chown -R sysadmin:www-data {{deploy_path}}");
    })->desc('Permissao SysAdmin');

task('deploy:limpando', function () {
    run("sudo chown -R www-data:www-data {{deploy_path}}");
})->desc('Permissao WWW-DATA');

after('deploy:prepare','deploy:preparando');
after('deploy:update_code','deploy:shared_rename');
after('cleanup','deploy:limpando');

task('deploy', [
    'deploy:prepare',
    'deploy:release',
    'deploy:update_code',
    'deploy:shared',
    'deploy:writable',
    'deploy:symlink',
    'cleanup',
])->desc('Deploy your project');
after('deploy', 'success');

Someone can help me? please

[antonmedv]

@murilodbva ->password('xxxxxx') does not supported in native (fixed in v5), use keys.

[murilodbva]

@antonmedv how can I do it? Or can you show me where can I look for?

[murilodbva]

Thanks @antonmedv
I'll wait for version 5

[marhub]

I have similar problem as @murilodbva .
I had keys configured with pass. With native, deployer was asking for key pass each time. So i removed pass from key - and it still is asking for password. Empty password is not working...

server('name', 'domain', 123) 
    ->user('user')
    ->stage('staging')
    ->identityFile(null, '../www_no_pass.ppk')
    ->set('deploy_path', '/home/www/site')
    ->set('branch', 'develop');

output is

➤ Executing task deploy:prepare
Enter passphrase for key '../www_no_pass.ppk':

[leo108]

@antonmedv will you support identity file with a passphrase in v5?

[antonmedv]

@leo108 if you wish.