Deployer: Is there a task to replace configuration parameters?

[dutchiexl]

I would like to replace some configuration parameters in yaml files, is there functionality present in deployer. Otherwise I would like to help to implement this.

[antonmedv]

It's cool feature. And i will implement it very soon in Deployer 3 release. Stay tuned.

[AlexStansfield]

Just wondered if this would also allow for parameter setup on initial deployment?

Composer will die on symfony2 installs if the db parameters are incorrect as it tries to connect to any configured DB when warming the cache.

This means on first deployment of a new code base the deployer will die unless the parameters in parameters.yml.dist are correct. Those who don't want to store db passwords in their repositories will run into problems.

Composer will actually prompt for values to put into parameters.yml on first install. I remember when I used to use capifony that it would relay these prompts back to the user.

Any chance this functionality will be adopted?

[antonmedv]

Yes, it's possible to implement this for Symfony2 deploy. Now Deployer runs composer with no-interaction key and this is good solution. But it's really easy to implement parameters.yml creation.

PS I deploy all my Symfony apps with Deployer, and deployment of new code base was ok, even without parameters.yml (Symfony don't connect to db when warming cache).

[AlexStansfield]

Ok, I'll look into the interaction thing.

But I just did a new deployment and composer failed because it couldn't find symfony db (what the db name was set to in parameters.yml.dist) in my mysql server.

When updated the db name to the correct one then it proceeds fine. Perhaps you already have the correct db credentials in the parameters.yml.dist.

[antonmedv]

Did you run migration tasks too?

[AlexStansfield]

Nope, there is no database migration to be done. Also as far as I'm aware you can't run migration before you've run composer as it needs all the dependencies composer downloads. And composer fails at cache stage.

[antonmedv]

Will try to reproduce it myself.

[AlexStansfield]

Perhaps it's something specific to things I've done in my entities. I'm using JMS serializer with doctrine.

[whyte624]

Yes, it's possible to implement this for Symfony2 deploy. Now Deployer runs composer with no-interaction key and this is good solution. But it's really easy to implement parameters.yml creation.

I didn't get it, can you please give an example? It would be very helpful for initial deploy.

[sadair]

I ran into the same thing. This article explains why doctrine try connect to db on cache clearing:

http://stackoverflow.com/questions/34023813/symfony-2-7-cacheclear-command-checks-every-database-connection

[cve]

Why it's run composer install without interaction event if I've override it like:

// dev server
server('dev', 'host', 22)
    ->user('user')
    ->identityFile()
    ->stage('dev')
    ->env('deploy_path', '/var/www/html/')
    ->env('branch', 'develop')
    ->env('env_vars', 'SYMFONY_ENV=dev')
    ->env('env', 'dev')
    ->env('composer_options', 'install')
;

?

[CharlyPoppins]

@whyte624 did you managed to implement parameters.yml creation ?

[whyte624]

@CharlyPoppins I'm creating build-time in a different place now.
More or less like that: https://github.com/octava/BranchingBundle/blob/master/Composer/ScriptHandler.php

[CharlyPoppins]

Ok thx I'll have a look

[danielsan80]

@elfet I'm not sure to understand.
The Symfony recipe run "composer install" with the --no-interaction option.
BUT this is not right for a general porpose because at the first deploy Composer needs to ask for the insersion of the parameters. (and when there are new parameters in parameters.yml.dist too)
We CAN remove --no-interaction from the composer_options
BUT it doesn't work
BECAUSE the run() function, used to run composer install, is not interactive.

Is there a way to have an interactive run()? (in Deployer\Server\Remote\NativeSsh)

Thanks

[antonmedv]

@danielsan80 yes, this will be possible in v5, see #1092

[danielsan80]

Ok,
so, waiting for v5, we can write a deploy:build-parameters task, to execute before deploy:vendors, who does the job of Incenteev\ParameterHandler\ScriptHandler::buildParameters. In this way composer install --no-interaction could be safe executed.

Here an example:

<?php
namespace Deployer;


use Symfony\Component\Yaml\Parser;
use Symfony\Component\Yaml\Yaml;
use Symfony\Component\Yaml\Inline;

task('deploy:build-parameters', function () {

    // Reset the tmp dir
    $tmpDir = '.tmp/deploy';
    exec('rm -Rf '.$tmpDir);
    @mkdir($tmpDir, 0775);

    // Download the parameters files
    try {
        download($tmpDir.'/parameters.yml', '{{release_path}}/app/config/parameters.yml');
        download($tmpDir.'/parameters.yml.dist', '{{release_path}}/app/config/parameters.yml.dist');
    } catch (\RuntimeException $e) {
        //Ignore exceptions
    }

    // At least parameters.yml.dist should be downloaded
    if (!is_file($tmpDir.'/parameters.yml.dist')) {
        throw new \InvalidArgumentException(sprintf('The dist file "app/config/parameters.yml.dist" does not exist.'));
    }

    $file = $tmpDir.'/parameters.yml';
    $distFile = $tmpDir.'/parameters.yml.dist';
    $exists = is_file($file);

    $yamlParser = new Parser();

    $action = $exists ? 'Updating' : 'Creating';
    writeln(sprintf('<info>%s the "%s" file</info>', $action, 'app/config/parameters.yml'));

    // Find the expected params
    $expectedValues = $yamlParser->parse(file_get_contents($distFile));
    if (!isset($expectedValues['parameters'])) {
        throw new \InvalidArgumentException(sprintf('The top-level key %s is missing.', 'parameters'));
    }
    $expectedParams = (array) $expectedValues['parameters'];

    // find the actual params
    $actualValues = array_merge(
        // Preserve other top-level keys than `$parameterKey` in the file
        $expectedValues,
        ['parameters' => []]
    );

    if ($exists) {
        $existingValues = $yamlParser->parse(file_get_contents($file));
        if ($existingValues === null) {
            $existingValues = [];
        }
        if (!is_array($existingValues)) {
            throw new \InvalidArgumentException(sprintf('The existing "%s" file does not contain an array', 'app/config/parameters.yml'));
        }
        $actualValues = array_merge($actualValues, $existingValues);
    }

    $actualParams = (array) $actualValues['parameters'];

    // Ask to the user for the missing params
    $isStarted = false;
    foreach ($expectedParams as $key => $message) {
        if (array_key_exists($key, $actualParams)) {
            continue;
        }

        if (!$isStarted) {
            $isStarted = true;
            writeln('<comment>Some parameters are missing. Please provide them.</comment>');
        }

        $default = Inline::dump($message);
        $value = ask(sprintf('%s:', $key), $default);

        $actualParams[$key] = Inline::parse($value);
    }

    $actualValues['parameters'] = $actualParams;

    // Save and upload the updated parameters.yml
    file_put_contents($file, "# This file is auto-generated before the composer install\n" . Yaml::dump($actualValues, 99));
    upload($tmpDir.'/parameters.yml', '{{release_path}}/app/config/parameters.yml');
});

than:

before('deploy:vendors', 'deploy:build-parameters');

[antonmedv]

In v5 it will be simple like this:

run('composer install', ['tty' => true]);

[Starojitski]

In v5 it will be simple like this:

run('composer install', ['tty' => true]);

Where do i put this line?

[CharlyPoppins]

I was wondering yesterday, found the command there :

//vendor/deployer/deployer/recipe/deploy/vendors.php
namespace Deployer;

desc('Installing vendors');
task('deploy:vendors', function () {
    if (!commandExist('unzip')) {
        writeln('<comment>To speed up composer installation setup "unzip" command with PHP zip extension https://goo.gl/sxzFcD</comment>');
    }
    run('cd {{release_path}} && {{bin/composer}} {{composer_options}}');
});

I think you have to override the command like this (not tested) :

task('deploy:vendors', function () {
    run('cd {{release_path}} && {{bin/composer}} {{composer_options}}', ['tty' => true]);
});

A set() parameter would have been simplier (contributors if you hear me :D)

[Starojitski]

@CharlyPoppins Thanks
I modified line 12 in local file ~/.composer/vendor/deployer/deployer/recipe/deploy/vendors.php
so that it looks now like this (note that mine has {{env_vars}} not found in your example)

run('cd {{release_path}} && {{env_vars}} {{bin/composer}} {{composer_options}}', ['tty' => true]);

It did not work for me. Instead of prompt for parameters, i still get An error occurred when executing the "'cache:clear --no-warmup'" command: exception.

[cedricverhaeghe]

To conclude, for people who want to be prompted for parameter.yml values when deploying Symfomy applications. You need to get rid of the --no-interaction flag and pass ['tty' => true] to the run command options.

An easy way is to override the built-in task like this:

desc('Installing vendors');
task('deploy:vendors', function () {
    if (!commandExist('unzip')) {
        writeln('<comment>To speed up composer installation setup "unzip" command with PHP zip extension https://goo.gl/sxzFcD</comment>');
    }
    $opts = '{{composer_action}} --verbose --prefer-dist --no-progress --no-dev --optimize-autoloader';
    run('cd {{release_path}} && {{bin/composer}} '.$opts.'', ['tty' => true]);
});