Deployer: Deploy freezes

[dfridrich]

| Q | A
| ----------------- | ---
| Issue Type | Question
| Deployer Version | 6.0.3
| Local Machine OS | macOS 10.13.1 (High Sierra)
| Remote Machine OS | Debian Jessie 8.5

Description

Deployer is stucked during deploy:prepare command. I have tried multiplexing and forwardAgent combination true or false with no effect.

Steps to reproduce

dep deploy

Content of deploy.php

<?php
namespace Deployer;

require 'recipe/symfony3.php';

// Project name
set('application', 'hourglass');

// Project repository
set('repository', '[email protected]:xxx/hourglass.git');

// [Optional] Allocate tty for git clone. Default value is false.
set('git_tty', true); 

// Shared files/dirs between deploys 
add('shared_files', []);
add('shared_dirs', []);

// Writable dirs by web server 
add('writable_dirs', []);

// Hosts
host('access.snackhost.eu')
    ->user('xxx')
    ->configFile('~/.ssh/config')
    ->identityFile('~/.ssh/id_rsa')
//    ->multiplexing(true)
//    ->forwardAgent(false)
    ->port(22)
    ->set('deploy_path', '/home/xxx/deploys/hourglass');

// Tasks

task('build', function () {
    run('cd {{release_path}} && build');
});

// [Optional] if deploy fails automatically unlock.
after('deploy:failed', 'deploy:unlock');

// Migrate database before symlink new release.

before('deploy:symlink', 'database:migrate');

Output log

dep deploy -vvv
[localhost] > export SYMFONY_ENV='prod'; git rev-parse --abbrev-ref HEAD
[localhost] < master
✈︎ Deploying master on access.snackhost.eu
• done on [access.snackhost.eu]
➤ Executing task deploy:prepare
[access.snackhost.eu] > export SYMFONY_ENV='prod'; echo $0
[access.snackhost.eu] < bash

[antonmedv]

Looks like your connected to server successfully:

[access.snackhost.eu] > export SYMFONY_ENV='prod'; echo $0
[access.snackhost.eu] < bash

But stuck after that. Can you give me access to your server to test it myself?

[dfridrich]

@antonmedv, I change timeout to 2s, this is what I get:

[Symfony\Component\Process\Exception\ProcessTimedOutException]                                                                                  
The process "ssh -A -p 22 -i ~/.ssh/deploy -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no [email protected]  'bash -s; printf "[exit_code:%s]" $?;'" exceeded the timeout of 2 seconds.

When I try to run this command directly from CLI I get connected.

[danielchodusov]

Having exactly the same problem. @dfridrich did you manage how to fix it?

[dfridrich]

@danielchodusov not yet, so I'm still using Capistrano :(

[antonmedv]

Sorry, for delay. Did you solve problem?

[dfridrich]

@antonmedv nope :(

[antonmedv]

Okay, lets try to investigate it. Please, contact me by email.

[stefax]

I have the same issue as dfridrich ... running into the timeout (300 seconds) for a symfony-recipe based deployment. The task executed is "deploy:vendors". It doesn't happen regularily, one time the timeout happens, another time it's ok.

[aryaNmlp]

Can i tell a web to ask this question Google help forum

[antonmedv]

@stefax set global timeout:

set('default_timeout', 3400);

[staabm]

I can see rsync calls freezing after 100% compied and no further progress. with ssh -vv you can see

...
[ci16] < debug2: channel 2: rcvd adjust 98426
[ci16] < debug2: channel 2: rcvd adjust 98438
[ci16] < debug2: channel 2: rcvd adjust 98403
[ci16] < debug2: channel 2: rcvd adjust 98443
[ci16] < debug2: channel 2: rcvd adjust 98575
[ci16] < debug2: channel 2: rcvd adjust 98948
[ci16] < debug2: channel 2: rcvd adjust 98877
[ci16] < debug2: channel 2: rcvd adjust 98453
[ci16] < debug2: channel 2: rcvd adjust 98527
[ci16] < debug2: channel 2: rcvd adjust 98480
[ci16] < debug2: channel 2: rcvd adjust 98681
     23,197,244 100%   15.73MB/s    0:00:01 (xfr#1, to-chk=0/1)
[ci16] < debug1: client_input_channel_req: channel 2 rtype exit-status reply 0
[ci16] < debug1: client_input_channel_req: channel 2 rtype [email protected] reply 0
[ci16] < debug2: channel 2: rcvd eow
[ci16] < debug2: channel 2: close_read
[ci16] < debug2: channel 2: input open -> closed
[ci16] < debug2: channel 2: rcvd eof
[ci16] < debug2: channel 2: output open -> drain
[ci16] < debug2: channel 2: obuf empty
[ci16] < debug2: channel 2: close_write
[ci16] < debug2: channel 2: output drain -> closed
[ci16] < debug2: channel 2: rcvd close
[ci16] < debug2: channel 2: send close
[ci16] < debug2: channel 2: is dead
[ci16] < debug2: channel 2: gc: notify user
[ci16] < debug2: channel 1: rcvd close
[ci16] < debug2: channel 1: output open -> drain
[ci16] < debug2: channel 1: close_read
[ci16] < debug2: channel 1: input open -> closed
[ci16] < debug2: channel 2: gc: user detached
[ci16] < debug2: channel 2: is dead
[ci16] < debug2: channel 2: garbage collecting
[ci16] < debug1: channel 2: free: client-session, nchannels 3
[ci16] < debug2: channel 1: obuf empty
[ci16] < debug2: channel 1: close_write
[ci16] < debug2: Received exit status from master 0
[ci16] < debug2: channel 1: output drain -> closed
[ci16] < debug2: channel 1: is dead (local)
[ci16] < debug2: channel 1: gc: notify user
[ci16] < debug2: channel 1: gc: user detached
[ci16] < debug2: channel 1: is dead (local)
[ci16] < debug2: channel 1: garbage collecting
[ci16] < debug1: channel 1: free: mux-control, nchannels 2
[ci16] < debug2: set_control_persist_exit_time: schedule exit in 600 seconds

[stefax]

@antonmedv thanks, i will try with increased timeout. Additionally, I'll debug with --log deploy.log. I just learned about this awesome option...

[antonmedv]

Also try to set controlpath to /dev/shm

[mkardakov]

Is there any update on it? I have also faced with a timeout issue

The process "ssh -A [email protected] 'bash -s; printf "[exit_code:%s]" $?;'" exceeded the timeout of 300 seconds.

This command should be fast due to it just retrieves a last command execution status

[barryvdh]

I'm seeing the same issue, when deploying on parallel to the same SSH user/host but different folder.

Host a1 is on server A, host b1 and b2 on the same server (same user/pass/etc as well), using the latest version (6.4.3)

The first 2 execute, the third is stuck.

+ /usr/bin/php7.1 bin/dep deploy staging -p -n -v --branch="$BITBUCKET_BRANCH" --revision="$BITBUCKET_COMMIT"
✈︎ Deploying revision e981a868db1beee4a33a511d3232cf8ca12d5640 on a1.example.com
• done on [a1.example.com]
✈︎ Deploying revision e981a868db1beee4a33a511d3232cf8ca12d5640 on b1.example.com
• done on [b1.example.com]
✈︎ Deploying revision e981a868db1beee4a33a511d3232cf8ca12d5640 on b2.example.com
• done on [b2.example.com]
➤ Executing task deploy:prepare
• done on [a1.example.com]
• done on [b1.example.com]

When I remove the -p flag, it works again (with exact same config using Bitbucket pipelines). Never seen this issue without the -p flag.

[eleftrik]

Some problem here when using -p. All my hosts (environments) are on the same server.
If I omit -p, I can deploy on all hosts, using dep deploy --roles=all.
Any news about this issue? Thank you

[barryvdh]

So perhaps an issue with re-using the connection on the same host?

[antonmedv]

I'm closing this issues as inactive. Please, feel free to open new issues.