Dotenv: 🔐👻 Multiple .env files, encrypting secrets, and committing .env to code

[lorensr]

I'm thinking it would be helpful to many for this section of the docs to be expanded/clarified.

https://github.com/motdotla/dotenv#should-i-have-multiple-env-files

Let's say I have a .env file with development values, and it's working as I'm developing with npm run dev. Now I want to do npm run deploy, but the .env values need to be different in production. How do I not have a separate .env file? By editing my single .env file every time I deploy? 😄

[GeorgeTaveras1231]

@lorensr so funny. I was just reading the docs and was just as confused by this section, and was going to open a similar issue 😄.

My interpretation of what the author meant is that you should avoid having a hierarchy of env files that are combined for a single "deploy". The message seems to be to maintain a clear separation between configs used for each "deploy".

Whether you do this by maintaining multiple files in your repo such as .env.development, .env.staging, etc. OR by having a single .env file that gets added to the project during the deployment process, I think should be left up to the developer.

On a project I'm currently working on, I've chosen to maintain all of the env vars in the repo, to make it easier to maintain and get the benefits of version control. Given that there are secrets in my env vars, I've chosen to have two env files for each deployment environment, one public.env (for env vars that don't have sensitive information (like UNION_DEPLOYMENT_ENV -- and env var that tells our internal libraries which deployment environment to target when referencing shared assets), and a private.env which is encrypted using a secret key we keep out of the repo.

So my project looks something like

/config
  /development
      /public.env
      /private.env.asc
  /qa
      /public.env
      /private.env.asc
  /production
      /public.env
      /private.env.asc

I'm not familiar with the 12 factor app, but I'd be curious to hear more about wether or not this approach is compatible with it's philosophies, and why.

[giacomocerquone]

Well, we can absolutely say that it is a confusing section of the readme. I was about to open the same issue and yes, it meant what @GeorgeTaveras1231 said.
you should not be sharing values between environments, so it's saying that even if you have a variable with an equal value in env.prod and env.stage, that must be defined two times, each in every single file because you'll inject just one of those two files into your build.

At the same time it's confusing because it's pretty common to direct a specific env file based on a certain deploy as the OP asked. Now, again, george is right, it's left to the developer but it's also sad that this package doesn't provide a more idiomatic way to do so.

Coming to the answer, I do not believe that this sentence is correct:
Whether you do this by maintaining multiple files in your repo such as .env.development, .env.staging, etc. OR by having a single .env file
There shouldn't be a "OR" there. Those two sentences are not an alternative:

• the first one answers about the way you store those
• the second one is the real answer to the OP question and it isn't a great way to do it.

In my opinion you should keep them all in your repo, commit not a public.env but an example.env that contains also the private keys but not the private values (because if you don't remember what values to put in there you can see the keys, otherwise you'd need to check the code) but this is just a matter of preference, depends on the team, etc.
And to direct you can use an args[0] value passed in the cli when starting the script or a "mother" env var you set before spawning the script: something like require('dotenv').config({ path: process.argv[2] }) for the first example and require('dotenv').config({ path: process.env.ENV_FILE }) for the second

[spawn-guy]

hello, i started to this this new "flavor" of naming .env files. so, at first i'd like to figure out which pattern to stick to: good-old .env{.environment} or the one that both of you are mentioning {environment}.env ? i see mentions of the latter in nestJS framework documentation.
am i missing some shift in global vision on how to name [dot]env files? (it clearly says dot in the beginning)

[spawn-guy]

then. we started to notice on some macBooks here.. the .env.local is not picked up at all, when .env file exists.
is this some breaking change? do i need to update my ci/cd pipelines that add only .env.local file on build time?

do we need to shift from

• " .env file with NO private values in the repo as an example with some defaults. which is overridden by .env.local by the developer"
  to
• "store .env.example file in repo with NO private credentials and then copy it to .env or put the full .env from external source"?

[motdotla]

This section is indeed confusing.

Can I ask though.. in regards to the example of npm run deploy where the "values need to be different in production". Haven't those already been set on the production server? Why would you need a .env file in production?

[morficus]

Haven't those already been set on the production server? Why would you need a .env file in production?

I was about to respond with this same question.

environment variables are exactly that... variables with values that can change based on the environment they are in. What everyone here seems to be trying to do is use the .env file as some type of system config file rather than a simple convenience during development.

Typically things such as DB passwords, API keys, JWT secrets and other sensitive information are stored as environment variables. It would be a HUGE "no no" to commit things like that to a repository... especially for production systems.

[macgyver]

Another confusing scenario is for configuration which is common to some environments but not all - for example if our application relies on a service that has its own environments, I'd like to be able to push updates to the URL for that service to all developers working locally without having to send a message to all of them and tell them to update their .env files - I think the use case could be summarized as public env vs. private env.

The docs seem to suggest that .env should only be used for values that are private to an individual env, but if that's the case I think it should be communicated more clearly (also if that's the case I think it significantly reduces the usefulness of dotenv)

[motdotla]

I'm not familiar with the 12 factor app, but I'd be curious to hear more about wether or not this approach is compatible with it's philosophies, and why.

@GeorgeTaveras1231 it is and it it isn't. By strict definition it is not, but, in the spirit of the 12 factor philosophy, it is because you encrypt the secret configs [a]

For what it's worth, I think your solution is elegant. Though, I also imagine it required you to write a fair bit of extra code to encrypt/decrypt, check environment, etc.

Zooming out a bit...

dotenv is a great library and has improved security for our profession. But there are 2 side effects to this I have seen (No good deed goes unpunished 😀)

1) many still do instinctively commit to code. It's natural for a developer and it's difficult to unlearn that behavior. The developer is not necessarily wrong here. apt: [b]

2) If you make a change to your .env file you have to notify ALL developers on your team to update their prospective .env files locally. Most shrug this off as just necessary to a twelve factor app but it is kind of a hassle when you think about it. And maybe it is avoidable. [c]

Has anyone seen what they are doing in the Ruby/Rails community around this problem? Rails has a new mechanism called credentials that takes an approach very similar to what @GeorgeTaveras1231 has done here.

If something like that would serve you well in your node repos, can you let me know?

cc @lorensr @macgyver

---

[a]

A litmus test for whether an app has all config correctly factored out of the code is whether the codebase could be made open source at any moment, without compromising any credentials.

https://12factor.net/config

[b]

https://99percentinvisible.org/article/least-resistance-desire-paths-can-lead-better-design/

[c]
The mechanism I use with my team (like @giacomocerquone does above) is a .env.example file that is committed to code and instructions in a README to run cp .env.example .env on their local machine. Then I or an infrastructure engineer also has to go configure the ENVs on staging and production environments.

(edited to separate footnotes from content more clearly)

[lorensr]

@motdotla Encrypted env files looks good for addressing 1 and 2.

My scenario is more basic—a single developer developing and deploying from the same directory. Would be nice if current deployment process:

cp .env.production .env
npm run deploy
cp .env.dev .env

didn't involve copying, eg by dotenv automatically using .env.[dev|staging|production] based on NODE_ENV.

I guess this is also relevant to the encryption scenario—dotenv deciding which file to decrypt & use based on NODE_ENV.

[motdotla]

@lorensr can I ask, are you deploying to anything like Heroku? Because almost all infrastructure providers have a mechanism to set environment variables. Then all you would have to do is: npm run deploy.

That said, yes, once you start committing your .env to git, your point here is well taken:

I guess this is also relevant to the encryption scenario

... because to a large degree the 12 factor app config methodology is about protecting secrets. Once those secrets are encrypted it's pretty much a moot point whether they are stored in code or in server memory. In fact, it's debatably more secure when encrypted as a file than in plain text in server memory.

But for the record, that is strongly discouraged with the current implementation of dotenv. Do not commit your .env to git. Set your ENV variables on your remote server. Here's an example from Heroku. https://devcenter.heroku.com/articles/config-vars

[macgyver]

[c]
The mechanism I use with my team (like @giacomocerquone does above) is a .env.example file that is committed to code and instructions in a README to run cp .env.example .env on their local machine. Then I or an infrastructure engineer also has to go configure the ENVs on staging and production environments.

This works great for initial setup but do you have a procedure for broadcasting changes or additions to the sample config to developers who have already set up their environments?

[motdotla]

A rudimentary one:

developers see the PR or git commit with changes to the .env.example file.

Or, they do not see the git change, pull the latest, and run the app or run specs and something breaks locally. Their first inclination (hopefully) is to look in the .env.example file for any recent changes.

Neither of these scenarios are bulletproof though. But so far the Ruby/Rails community's new approach has been (at least in terms of solving this problem you describe).

[macgyver]

Thanks for the consideration! I think you realize these are pretty unsatisfactory solutions, especially for larger teams where I wouldn't expect every developer to review every pull request, and certainly wouldn't want to waste everyone's time as each developer struggles to discover why their system broke when they rebased.

Our team uses 1password to store shared secrets, and I know they have a command line API, so maybe we can do something similar to the Ruby/Rails credentials thing to automate this.

[DanielRuf]

This is a problem, create-react-app and others support this. Is there another package which handles all these (.env.local) or do we have to check in a sample .env and run the cp / mv command as needed (even symfony supports this thanks to the dotenv PHP package)?

[giacomocerquone]

@DanielRuf I recommend you my blog post about this matter: https://giacomocerquone.com/blog/rethinking-env-vars-in-react-native. I believe this is as far as we can get

[DanielRuf]

I am aware of this and such solutions but we already have a React (not React Native) application in a monorepo and a backend and use dotenv for the backend part because of credentials.

Your solution is just a workaround for me. But thanks for pointing me to it.

[giacomocerquone]

@DanielRuf
Many people requested a solution like yours and I can provide such one.
One way would be to use a similar solution showed in my blog post, but with a nodejs script that writes what's inside the process.env variable to that file showed in my blog

[DanielRuf]

That is not what most of us want or need but thanks.

An optin feature which enables this directly in this library would make sense. Otherwise I think most of us would write a small node script like I did for the frontend part to copy the env file on npm build. But then this library would be useless.

[alejo4373]

As @lorensr I have a similar use case where my development and test environments are the same (my laptop). I run a few integration tests agains my back-end that involve my database. I store my DATABASE_URL in my .env file but I need to change the url when testing. I would love to pick what .env file to load based on NODE_ENV. I know a few of you mentioned a few workarounds but it would be cool if there was a way to accomplish this with this package.

Anyone has any suggestions as to how to address my use case maybe differently? That doesn't polluting my code with if statements or hardcoded values?

@motdotla Encrypted env files looks good for addressing 1 and 2.

My scenario is more basic—a single developer developing and deploying from the same directory. Would be nice if current deployment process:

cp .env.production .env
npm run deploy
cp .env.dev .env

didn't involve copying, eg by dotenv automatically using .env.[dev|staging|production] based on NODE_ENV.

I guess this is also relevant to the encryption scenario—dotenv deciding which file to decrypt & use based on NODE_ENV.

[chuckxD]

@alejo4373

I store my DATABASE_URL in my .env file but I need to change the url when testing. I would love to pick what .env file to load based on NODE_ENV.

If I'm not mistaken you should just be able to pass --env [dev,production,staging,etc] on any command in order to pull in your variables for that specific environment. I can't remember if dotenv supports this flag out of the box but used it all the time on my previous serverless project, see serverless dotenv plugin usage here.

Alternatively, you should just be able to prepend setting your NODE_ENV value or any other value you may want to override in your .env file prior to running whatever you wanna run, e.g. NODE_ENV=test npm run start and the command should pick up .env.test instead of some default fallback.

When all else fails or to handle something that doesn't make sense for dotenv practices, I will usually fall back on env-cmd or pass through specific env parameters/variables from npm-run-script with -- flag e.g. npm run start -- export DATABASE_URL=foo.

Maybe that helps, most cases you shouldn't have to fight dotenv if you're using it as intended, I suppose there's also dotenv-flow but not 100% certain on the specific differences between the two packages

[zemlanin]

I run a few integration tests agains my back-end that involve my database. I store my DATABASE_URL in my .env file but I need to change the url when testing

@alejo4373 I've solved this "same machine, separate .env for testing" problem by:

1. mimicking env-options.js in my application code:
   js require("dotenv").config({ path: process.env.DOTENV_CONFIG_PATH || null, });
2. and running tests with env DOTENV_CONFIG_PATH=./test/.env npm test command

It's more wordy to run, than npm test -- --env [dev,test,production] or env NODE_ENV=test npm test, but:

• takes less code to implement;
• doesn't invent any new conventions;
• more honest, i guess

[plashenkov]

Having .env, .env.local, etc. is a common and sensible practice:

• https://symfony.com/doc/current/configuration.html#overriding-environment-values-via-env-local (Symfony)
• https://nextjs.org/docs/basic-features/environment-variables#default-environment-variables (Next.js)
• https://create-react-app.dev/docs/adding-custom-environment-variables/#what-other-env-files-can-be-used (Create React App)

The mechanism I use with my team (like @giacomocerquone does above) is a .env.example file that is committed to code and instructions in a README to run cp .env.example .env on their local machine. Then I or an infrastructure engineer also has to go configure the ENVs on staging and production environments.

I used the same approach before but it is just less practical.

As for "12 factor app philosophy"... I really believe we should not follow blindly.

[mdoliner-st]

I recently learned about https://www.npmjs.com/package/dotenv-flow, which might address some people's needs.

[yahmad3495]

I am working on a Twitter developer app with a partner and need to load in my API keys/tokens through a .env file. However, my partner has already established/uploaded her .env files in the code. At this point, she is able to run the code fine, but I still can't because my .env hasn't been uploaded. Is there a way where the code can tell which .env file it's referencing so I can run the code as well?

[quantuminformation]

I use multi env files all the time with firebase which has a custom deploy process and hides the complexity of storing the .env file on the server, --vite supports multi env's out the box.

[mattcarlotta]

I noticed that this package seems to be relatively inflexible when dealing with multiple .env files across multiple environments (and it hasn't been updated in over a year), so instead of writing a custom file and/or scripts to handle this, I wrote a custom package that not only offers the same features as dotenv with dotenv-expand, but is focused on flexibility and performance with the smallest footprint possible.

If anyone is interested... PRs and feedback are welcomed.

About the package:
✔️ 0 Dependencies
✔️ Between 40-70% faster than dotenv+dotenv-expand
✔️ Compiled and minified for ES5+
✔️ Written in Typescript and comes bundled with Typescript types
✔️ Unopinionated .env file naming
✔️ Loads files from a custom directory
✔️ Loads one or many .env files simultaneously
✔️ Preloads one or many .env files simultaneously
✔️ Same interpolation as dotenv-expand, but also supports command line substitutions
✔️ Supports default values for undefined interpolations
✔️ Optional overriding of Envs in process.env

What it doesn't cover:
❌ Command line default keys: $KEY:-default -- no plans to implement, but willing to if it doesn't impact performance
❌ Multi-line values within keys -- no plans to implement (base64 encoded strings seem more appropriate)
❌ Referencing $KEYs across .env files -- no plans to implement

Documentation and Source Materials

Instructions
Repo
BundlePhobia
NPM

Click to expand package stats

Build stats

| package | files | unpacked | minified | gzipped | download time | coverage |
| --- | --- | --- | --- | --- | --- | --- |
| snackables | 5 | 25.1kB | 1.9kB | 1kB | 34ms (2G) / 21ms (3G) | 100% |
| denv+denv-exp | 18 | 39.0kB | 1.8kB | 1kB | 35ms (2G) / 21ms (3G) | N/A |

Tests

Some rudimentary tests.

Loading and interpolating a single small env file:
| package | type | iterations | duration (3 fastest runs out of 6) | avg | fastest |
| --- | --- | --- | --- | --- | --- |
| snackables | default (.env) | 500,000 | 13.58s, 13.65s, 13.70s | 13.64s | 100% |
| denv+denv-exp | default (.env) | 500,000 | 23.61s, 23.91s, 24.01s | 23.84s | 57.21% |

Loading and interpolating a single large env file:
| package | type | iterations | duration (3 fastest runs out of 6) | avg | fastest |
| --- | --- | --- | --- | --- | --- |
| snackables | .env.interp only | 5,000 | 22.49s, 22.54s, 22.56s | 22.53s | 100% |
| denv+denv-exp | .env.interp only | 5,000 | 72.14s, 72.15s, 72.63s | 72.31s | 31.16% |

Loading and interpolating multiple small env files (1, 2, 3, 4):
| package | type | iterations | duration (3 fastest runs out of 6) | avg | fastest |
| --- | --- | --- | --- | --- | --- |
| snackables | .env, .env.development, .env.local, .env.development.local | 500,000 | 25.88s, 25.96s, 26.00s | 25.95s | 100% |
| denv+denv-exp | .env, .env.development, .env.local, .env.development.local | 500,000 | 41.59s, 41.63s, 42.05s | 41.76s | 62.14% |