Deployer: Support .env files (dotenv)

[staabm]

| Q | A
| ----------------- | ---
| Issue Type | Feature Request
| Deployer Version | 6.0.3
| Local Machine OS | Ubuntu16 lts
| Remote Machine OS | Ubuntu16 lts

Description

recipes tend to duplicate config settings which are sometimes already defined elsewhere in the project.
this might be e.g. a url to the repository, usernames, passwords etc.

there is some kind of standard which solves such problems. usually you use a dotenv file,
see e.g. https://symfony.com/components/Dotenv

that way you have a config file in a format consumable by the application itself, its clitools or other related parties. the files are technologie and programming language/framework agnostic so information placed there can be used in a lot of places, without duplication.

would be great to get some kind of out-of-the-box support for dotenv files.

[Blizzke]

If all you want is the values from your .env available in templates (eg {{APP_NAME]}) etc, that is something you can solve quickly in about 3 lines of text.

First of all: add Symfony's dotenv as a requirement:
composer require symfony/dotenv:"^3" (Deployer is still on 3.x so I'll use that)

When that is done, add these 2 somewhere in your deploy script (I usually do it right after the namespace):

(new \Symfony\Component\Dotenv\Dotenv())->load('.env');
array_map(function ($var) { set($var, getenv($var)); }, explode(',', $_SERVER['SYMFONY_DOTENV_VARS']));

That's it. All variables from your .env are now available as Deployer variables.

If you want the variables from the remote .env, that is also fairly easy to achieve, but you'll need a connection to the remote host (obviously), so that can only happen after those were setup:

$environment = run('cat {{deploy_path}}/shared/.env');
$dotenv = new \Symfony\Component\Dotenv\Dotenv();
$dotenv->populate($dotenv->parse($environment));
array_map(function ($var) { set($var, getenv($var)); }, explode(',', $_SERVER['SYMFONY_DOTENV_VARS']));

[leeuwd]

For others struggling with this:

For Laravel:

composer require vlucas/phpdotenv

And put this in the top of your deploy.php:

// Load .env
with(new \Dotenv\Dotenv(__DIR__))->load();

[Blizzke]

If you are running deployer from within a laravel project, you don't need to run the require, it's there already.

I did realise, hence the second part of my message, during a deploy you'd probably want the settings from the remote server, not the local one. Unfortunately phpdotenv doesn't seem to support loading from a string, it needs a file as source. So even for a laraval project you'd have to include Symfony's dotenv if you want to load from remote.

[cilefen]

Here is a simple way, building on what @Blizzke wrote above (which is good stuff, thank you!):

task('deploy:dotenv', function() {
    $environment = run('cat {{deploy_path}}/shared/.env');
    $dotenv = new \Symfony\Component\Dotenv\Dotenv();
    $dotenv->populate($dotenv->parse($environment));
})->desc('Load DotEnv values');
after('deploy:shared', 'deploy:dotenv');

[cilefen]

Also, to anyone using Symfony with DotEnv, it will pay to scrutinize the bin/console command:

if (!isset($_SERVER['APP_ENV'])) {
    if (!class_exists(Dotenv::class)) {
        throw new \RuntimeException('APP_ENV environment variable is not defined. You need to define environment variables for configuration or add "symfony/dotenv" as a Composer dependency to load variables from a .env file.');
    }
    (new Dotenv())->load(__DIR__.'/../.env');
}

Remove the outer conditional if you plan on using DotEnv in production while calling bin/console. With that done, what I wrote in the comment above is unnecessary.

[antonmedv]

Yes, actually this is correct solution.

[staabm]

Great to see this examples. I guess we could close this issue and the people who need .env can use those snippets posted above.

@antonmedv do you agree that we _dont_ need a more deployer-core like approach for this?

[antonmedv]

What is deployer-core like approach ?

[cafferata]

I think @staabm mean including the above code snippet as a standard task in the deployer codebase. As a result, other people do not have to duplicate this task code on the various projects.

[antonmedv]

This can be done. Can anyone create PR?

[inverse]

Thanks @Blizzke + @cilefen. Reworked what you guys did to load the environment for deployment and came up with this.

task('load:env-vars', function() {
    $environment = run('cat {{deploy_path}}/shared/.env');
    $dotenv = new \Symfony\Component\Dotenv\Dotenv();
    $data = $dotenv->parse($environment);
    set('env', $data);
});

before('deploy', 'load:env-vars');
before('rollback', 'load:env-vars');

Which hooks in before deploy and post rollback.

[chapterjason]

With Symfony recipe the .env is always empty and not working like expected.

I'm expecting something like this

$new_variables = run('cat {{release_path}}/.env.dist');
$old_enviroment = run('cat {{deploy_path}}/shared/.env');
array_merge($dotenv->parse($new_variables), $dotenv->parse($old_enviroment), get('env'));

And then the output will be save to {{deploy_path}}/shared/.env to take effect of the changes from host.

// EDIT
Currently I use the following, thanks to the others for the inspiration!

task('load:env-vars', function () {
    $dotenv = new Dotenv();
    $new_variables = run('cat {{release_path}}/.env.dist');
    $old_enviroment = run('cat {{deploy_path}}/shared/.env');
    $environment = array_merge($dotenv->parse($new_variables), $dotenv->parse($old_enviroment), get('env'));
    $dotenv->populate($environment);
    set('env', $environment);

    $loadedVars = array_flip(explode(',', getenv('SYMFONY_DOTENV_VARS')));
    unset($loadedVars['']);
    $loadedVars = array_flip($loadedVars);
    $first = true;
    foreach ($loadedVars as $loadedVar) {
        if ($first) {
            run('echo "' . $loadedVar . '=' . $environment[$loadedVar] . '" > {{deploy_path}}/shared/.env');
            $first = false;
        } else {
            run('echo "' . $loadedVar . '=' . $environment[$loadedVar] . '" >> {{deploy_path}}/shared/.env');
        }
    }
});
before('deploy:shared', 'load:env-vars');

[cilefen]

@chapterjason That is an interesting idea. I think that this issue is focused on getting .env files parsed and read in the first place. What you are proposing could be a good follow-up. As far as I know Symfony via DotEnv outside of the Deployer context no longer updates the .env file when .env.dist changes so this would be an added feature.

[chapterjason]

@cilefen Yeah thats right, that was a good side-effect of my idea.

The env variables are changing in my workflow by host, so that I can specify them there and they will get overriden and taken care of old and new env.

But what is with a rollback? Then the .env shouldn't be shared, cause of different used .env, some of them could be someday be deprecated, get noticed by the system and it will throw everytime a warning.

With a new release, it should work like this. (tested and working)
The .env should be removed from shared_files
```php
namespace Deployer;

use Symfony\Component\Dotenv\Dotenv;

task('deploy:env-vars', function () {
$dotenv = new Dotenv();
$environment = [];

// Add .env.dist from previous_release
if (has('previous_release')) {
    if (test('[ -f {{previous_release}}/.env.dist ]')) {
        $environment = array_merge($environment, $dotenv->parse(run('cat {{previous_release}}/.env.dist')));
    }
}

// Add .env.dist from current release
if (test('[ -f {{release_path}}/.env.dist ]')) {
    $environment = array_merge($environment, $dotenv->parse(run('cat {{release_path}}/.env.dist')));
}

// Add .env from previous release
if (has('previous_release')) {
    if (test('[ -f {{previous_release}}/.env ]')) {
        $environment = array_merge($environment, $dotenv->parse(run('cat {{previous_release}}/.env')));
    }
}

// Add host env
$environment = array_merge($environment, get('env'));

// Populate and update env
$dotenv->populate($environment);
set('env', $environment);

// Write .env to the current release
reset($environment);
$first = key($environment);
foreach ($environment as $key => $value) {
    run('echo "' . $key . '=' . $value . '" '.(($first === $key) ? '>' : '>>').' {{release_path}}/.env');
}

});

// After deploy:update_code, cause of the updated .env.dist
after('deploy:update_code', 'deploy:env-vars');
````

[chrispage1]

Just throwing in my 2 cents here for those of you that use Laravel. It's all well and good loading the variables from the remote environment but for those of you that use Laravel Mix you'll run into issues as it'll always load from the local .env file.

As a 'fix' to this issue, I've created a set of tasks that enable you to replace your local .env file with the remote for the duration of the deployment. I'd welcome any feedback for this one as I know it's not perfect but for me at least this works pretty well!

The code

// Task to load in our remote .env file for local compilation
task('env:load', function () {
    runLocally('cp .env .env.deployer');
    download('{{deploy_path}}/shared/.env', '.env');
})->desc('Load the contents of our remote .env file so we can compile with it locally');
after('deploy:shared', 'env:load');

// Task to restore our local env file once it has been changed by env:load
task('env:restore', function () {
    if(test('[ -f ./.env.deployer ]')) {
        run('mv .env.deployer .env');
    }
})->desc('Recover local environment file after env:load has been used')->local();

// Build our assets locally
task('npm:build', function () {
    write('Compiling assets locally for {{stage}}. This can take a while...');
    runLocally('npm run prod -s');
})->desc('Compile npm files locally');
after('deploy:shared', 'npm:build');
after('npm:build', 'env:restore');

// Upload our locally compiled assets
task('deploy:upload-public', function () {
    upload('public', '{{release_path}}');
})->desc('Upload the public directory to the environment');
after('npm:build', 'deploy:upload-public');

Explaining the process

So during a standard deployment procedure it works just like this -

1. Make a copy of our local .env file so we can recover it later
2. Download the remote .env replacing our local file so we can realistically compile our npm/webpack files locally.
3. Run npm production build, this will take a short while so I've added appropriate logging in here
4. After npm build, restore our local machines environment file.
5. Upload public directory (where our compiled assets will be) to the remote environment
6. Given that all goes well, processing will continue and you'll have a successfully deployed Laravel environment :)

If your deploy fails, you can always run env:restore again too, there's no harm in running it just in case! - after('deploy:failed', 'env:restore'). I've made each task as modular as possible so that you can just run an individual task without it having to be run as a chain.

Just for the record, I think Deployer is awesome, love the flexibility and so glad I came across it!

[simioluwatomi]

For others struggling with this:

For Laravel:

composer require vlucas/phpdotenv

And put this in the top of your deploy.php:

// Load .env
with(new \Dotenv\Dotenv(__DIR__))->load();

This fails in new Dotenv versions. This works for me though

namespace Deployer;

\Dotenv\Dotenv::create(__DIR__)->load();

[ryancwalsh]

I had to run this instead: require __DIR__ . '/bootstrap/autoload.php'; \Dotenv\Dotenv::create(__DIR__)->load();

[bretvanhorn]

For others struggling with this:

For Laravel:

composer require vlucas/phpdotenv

And put this in the top of your deploy.php:

// Load .env
with(new \Dotenv\Dotenv(__DIR__))->load();

This worked well for me—note that you simply use the env('VAR_NAME') function to get your variable.

[ludo237]

For others struggling with this:
For Laravel:
composer require vlucas/phpdotenv
And put this in the top of your deploy.php:

// Load .env
with(new \Dotenv\Dotenv(__DIR__))->load();

This fails in new Dotenv versions. This works for me though

namespace Deployer;

\Dotenv\Dotenv::create(__DIR__)->load();

I suggest createImmutable instead. Both work fine tho

[antonmedv]

Support added in 36596a618e73af764cbc93fa913a960115bac6c3
php set('dotenv', '{{current_path}}/.env');

[loevgaard]

For people finding this on Google and wanted a solution to update your environment variables on your remote server I created a library for that: https://github.com/Setono/deployer-dotenv