Vercel: Spec out new env system

[leo]

After zero-config, we want to solve our environment variable situation.

The main problem right now is that there are two different ways of using env in now dev and now. We should suggest just one (this could be solved by having a "Dev" toggle in the UI for environment).

Furthermore, you currently have env and build.env.

Also, staging/prod and dev are are problem in general:

https://twitter.com/bitr0t_/status/1152370091155214336?ref_src=twsrc%5Etfw%7Ctwcamp%5Etweetembed&ref_url=https%3A%2F%2Fwww.notion.so%2Fzeithq%2F7a16d63fa31245bab8a7899a7d47f5a7%3Fv%3D1947f17e724d4ee4bc1d0aa95dab2614%26p%3Ddbb63b295d2845918211258cd92a6d8f

[leo]

Today, we received the feedback that I'd be great to set these right in the UI for a project.

Basically, you can set every secret and add a dev AND/OR prod value for it. After that, now dev will automatically download the value. You wouldn't even need now.json for env anymore.

For existing deployments (so that reverts work) we want to lock to the env that was available at that exact time.

[ctrlplusb]

Glad to hear this is getting some attention. Could I also ask that "test" cases be considered. i.e. multiple env sets for a local machine

[enzoferey]

I have been struggling for hours to get my environment variables working in the nextjs template provided my Zeit now.

I think the docs are pretty confusing as it points out to different ways of exposing environment variables. The docs I have looked at:

• Serverless Function Environment Variables and Secrets
• Environment Variables and Secrets
• Exposing configuration to the server / client side

---

Eventually I got it working like this:

now.json =>

{
  "env": {
    "TEST": "@test"
  }
}

.env =>

TEST="test-dev"

next.config.js =>

{
  "env": {
    "TEST": process.env.TEST
  }
}

anywhere on my functions =>

console.log(process.env.TEST); // "test-dev"

That's for dev environment.

For production you need to set Now's secrets executing:

now secrets add test "test-prod"

Then you will see "test-dev" when running now dev and "test-prod" when deployed running now.

Note, works as well for build.env.

---

Edit: never mind, I keep having conflicts and when logging process.env and process.env.TEST right after each other I see different values 🤔

[varna]

@enzoferey variables like process.env.TEST oftenly get replaced during "bundle time" if certain plugin is used (forgot its name thou, most frameworks use it). So process.env actually doesn't exist, it's just that process.env.TEST gets replaced with .env file value.

[enzoferey]

@varna I guess you meant dotenv ? I wasn't using it as we are supposed to don't need it using the now.json solution.

[varna]

Yes, I meant dotenv. I think most SPA frameworks use it. But nevermind @enzoferey, I misread your comment 😭

[ties-v]

Today, we received the feedback that I'd be great to set these right in the UI for a project.

Basically, you can set every secret and add a dev AND/OR prod value for it. After that, now dev will automatically download the value. You wouldn't even need now.json for env anymore.

For existing deployments (so that reverts work) we want to lock to the env that was available at that exact time.

@leo Looks good to me! Maybe it is possible to add "custom" environments, so I can setup a dev, staging _and_ production environment as I wish ;)

[leo]

Thanks for the input, @ties-v! We'll involve this feedback in our upcoming discussions about refining the environment variable system.

[dohomi]

I am trying out the zero-config as well and I don't understand how to define ENV variables without attaching a now.json file. Did I overlook something?

[leo]

@dohomi There's currently no way to do that.

You will have to add env in now.json. But as explained above, we're already working on a solution!

[dohomi]

@leo thanks for the heads up - I thought I forgot something while reading the docs.

One question: if I want a "as near as zero config" at the moment, I started to have an "api/[my-service-file.js]" structure. What would be an ideal now.json file to handle all the incoming requests and having the least migration if the zero config evolves?

[vadistic]

I did not feel like retyping secrets this morning... I present you now-dotenv It use Now API to sync your Now Secrets with .env files.

You can check it out as workflow idea. Dotenvs are easy, but secrets are must-have because of github CI/ sharing/ safety. Here I have something like the best of two worlds.

Btw. It would be nice to allow some unsafe/dev secrets that can be read via API. Something safer than commiting .env to git, but shareable.

[lorensr]

dotenv is very widely used at this point. I think it would be nice if Now supported .env files

[ties-v]

Food for thought: I stumbled upon SecretHub which is a framework for secret management for all microservices in your stack, independently of the technology used.

To use such system with Now, a integration system for secrets would be required. Maybe this is something that can be taken into consideration?

[pvdlg]

At a minimum it would be useful to be able to pass an argument to now dev to indicate which .env file to use. Currently the path and name of the .env file is hardcoded: https://github.com/zeit/now/blob/d380902ad3490b5098bdd4a5e4d6f14c7aa7ccba/packages/now-cli/src/util/dev/server.ts#L709

My use case is that I have an API endpoint that I use in my app and want to be able to use now dev so my app either call the real API to debug/dev or a local Docker container to run unit tests.

As an example I would like to do the following.

In .env.dev

URL=https://some-api.com

In .env.test

URL=http://localhost:8080

Then I could start my local now app with either now dev --dotenv .env.dev to hit the real API or now dev --dotenv .env.test to hit the local Docker.

[JerryGreen]

My proposal is to force a convention which will cover most situations. Out of the box, NextJS should be reading some of these files, if present. This is especially convenient to use with Github integration. I use this convention myself in all my projects. In ascending order of precedence (from less important, to more important which will override previous), here it is:

1. .env.default
   This is full set of environment variables which are used by your NextJS app, with some default values (which often are empty strings; an example: SOME_MY_VAR= or equivalent SOME_MY_VAR='' or SOME_MY_VAR="", - in which case it doesn't set anything but it helps to indicate what's even possible to change with env vars). Always read by NextJS, if present. Should be committed into source version control.

2. .env.production
   This is some environments related to remote (deployed) app. Read by NextJS only in deployed apps or when next start is used. Should be committed into source version control. Overrides values of the file above.

3. .env
   This is custom settings for a concrete user. It's supposed to be in .gitignore and in many cases the file should be empty or even not exist. When debugging, though, you may override anything with this file. May have secret keys. Overrides values of the files above.

[agrohs]

Why not just use dotenv-flow?

On Dec 26, 2019, at 2:51 PM, Jerry Green [e-mail] wrote:

My proposal is to force a convention which will cover most situations. By default, NextJS should read some of these files, if present. In ascending order of precedence (from less important, to more important which will override previous). This is especially convenient to use with Github integration. I use this convention myself in all my projects. So here it is:

.env.default
This is full set of environment variables which are used by your NextJS app, with some default values (which often are empty strings; an example: SOME_MY_VAR= or equivalent SOME_MY_VAR='' or SOME_MY_VAR="", - in which case it doesn't set anything but it helps to indicate what's even possible to change with env vars). Always read by NextJS, if present. Should be committed into source version control.

.env.production
This is some environments related to remote (deployed) app. Read by NextJS only on deployed apps or when next start is used. Should be committed into source version control. Overrides values of the file above.

.env
This is custom settings for a concrete user. It's supposed to be in .gitignore and in many cases the file should be empty or even not exist. When debugging, though, you may override anything with this file. May have secret keys. Overrides values of the files above.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub https://github.com/zeit/now/issues/2613?email_source=notifications&email_token=AABHHB4DH5ARRGHMGKKTZHDQ2UKLJA5CNFSM4IGP3V62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHWEEWQ#issuecomment-569131610, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABHHB6PY5OHK3HMZV5BAPTQ2UKLJANCNFSM4IGP3V6Q.

[JerryGreen]

@agrohs not every part of that flow has sense, especially for NextJS. Also, they state to store canonical .env in source control, which is absolutely weird for developers who by some reasons prefer a single .env file over all of this.

[agrohs]

In that case, all I can share is that the convention you proposed is overly opinionated IMHO

On Dec 26, 2019, at 5:53 PM, Jerry Green [e-mail] wrote:

@agrohs https://github.com/agrohs not every part of that flow has sense, especially for NextJS. Also, they state to store canonical .env in source control, which is absolutely weird for developers who by some reasons prefer a single .env file over all of this.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/zeit/now/issues/2613?email_source=notifications&email_token=AABHHB3VB5M36NQRFLZE67LQ2U7WNA5CNFSM4IGP3V62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHWJCWA#issuecomment-569151832, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABHHB4A2HXCF6KXKU7W22DQ2U7WNANCNFSM4IGP3V6Q.

[leo]

Thank you for creating this issue!

Please note that – in order for us to be able to handle each feature request with care – we have to consume all of them through a unified pipeline that makes it easier for us to prioritize, track and progress on the features our user base is interested in.

In turn, the Issues tab on this repository is best only be used for reporting bugs, which we can immediately act on.

For well-explained feature requests (like the one you just posted), please contact us at [email protected], so that we can consider adding it to our roadmap.

In the case of your specific feature recommendation, you can rest assured that we're already tracking it on our roadmap and you can expect to see an update on our Twitter account or Blog soon.

[agrohs]

@leo, no issues with closing this to focus on Issues but where can we track the new solution that you guys have supposedly been working on for some time around env changes???

[JerryGreen]

I don't really get why closing. Maybe if we will see some solutions very soon, like within few days, - then I will understand this rush of closing the issue. But for now it's just weird seeing the issue is "just closed", without real reason.

First of all, am I only the one noticing the fact @leo thanked himself for creating his issue, and later, with excuses to himself, closed this? How weird is that? Haven't you noticed, @leo, you yourself created that particular issue?

I understand the concern of having issue number small. I also understand that ofc there may be some underlaying dev flows, outside of Github. Also, I really like your guys blog, but it seems it's not even possible to have an email subscription for it (unfortunately). Github Issues is one of the best places for tracking software updates, overall. Regarding bugs or features, whatever. Issues are essentially multipurpose, not just for bugs, - that's why they called issues, not bugs. There are tags for that, on github, and also very good search with keywords.

So I feel some dishonesty to the community 😒

[pke]

@JerryGreen That looks like an automated closing message. I saw the exact same text a couple of times now and I agree its frustrating. They are going from an open process like GH issues to a closed system via Email. Their support at the email address is very (!) quick to respond though.

However I'd like they keep tracking issues here instead of some internal tool (only).

On topic, I wonder why .env is not supported via dotenv already. Why go to all those lengths of discussion? And according to https://12factor.net/config there should be one, and only one .env file for local development. Easy. Isn't it? For everything else you use the Build/CI systems dynamic env vars.

I cam here cause I wondered why now dev insist on binding to 3000 instead of mine .env written one: PORT=3001

I am going to create a PR, lets see how it will be received.

[rkrueger11]

Hey @Leo - looks like this was accidentally closed? Any update on how the new env system is progressing?

[GriffinSauce]

@rauchg I feel like I've read half the internet by now and still have no clue how to set this up to work, let alone how to do it "properly".

Is there a comprehensive guide how to currently create multiple environments like dev/staging/prod?

[rauchg]

The canonical answer from us in the form of customizable per-environment ENV in our UI is shipping very very soon. Apologies for the delay on this.

[SpainTrain]

The canonical answer from us in the form of customizable per-environment ENV in our UI is shipping very very soon. Apologies for the delay on this.

Very exciting! Is there an issue to subscribe to for updates? Thanks!

[xmflsct]

Having similar problem when running now dev locally with Gatsby and serverless functions.

[JerryGreen]

@rauchg it might be more sense to keep this issue open while the functionality is not yet shipped to master branch

(I personally prefer this issue to be open, and only closed when the functionality is there, - otherwise it's just harder to track everything... There's a lot of issues, you know, - why just don't close them immediately? P.S. sorry for such an extravagant example though)

[treyhuffine]

Is there anywhere that we can track the progress for this or opt into a beta version for our deployments?

[JerryGreen]

Is there anywhere that we can track the progress for this

YES! I mean... No, there isn't. But this is what I'm whining about, in a message above your one. The issue should be open, to track if/when it's done.

or opt into a beta version for our deployments?

This is not an official answer, but no, there isn't. I'm very into this issue, I would know otherwise.

[alexbudure]

I believe you can opt in for beta here

http://zeit.co/_flags

[ddman]

I created two teams, and each one set secret with different value, and each one watch different git branch ,to solve this issue.

[mirshko]

@ddman i was thinking of doing the same.

enabling the feature flag works though!

just tested it out.

[treyhuffine]

Thanks for sharing @alexbudure! Do you know if the beta features also have documentation anywhere or is that only written after a release?

[laurensnl]

That's awesome @alexbudure! I've set up the environmental variables using the web UI. Works like a charm!

[xmflsct]

@alexbudure Thank you! But looks like my previously stored credentials are not showing up there. Does it mean that I need to create those ones on the web, and remove those @ from now.json?

[laurensnl]

@alexbudure Thank you! But looks like my previously stored credentials are not showing up there. Does it mean that I need to create those ones on the web, and remove those @ from now.json?

That's right. It's what I did and it worked.

Keep your .env file and require('dotenv').config() in next.config.js for running the dev server locally though.

[treyhuffine]

If I'm understanding this correctly, it only lets us create environment variables that distinguish between a production and a preview deploy.

However, if we wanted to have a staging branch that was set to a custom domain (staging.mysite.com), it would still use the production environment variables since this deploys as --prod with the github integration?

[mirshko]

If I'm understanding this correctly, it only lets us create environment variables that distinguish between a production and a preview deploy.

However, if we wanted to have a staging branch that was set to a custom domain (staging.mysite.com), it would still use the production environment variables since this deploys as --prod with the github integration?

nope!

basically all prod deploys if you use the github connection on the default branch will be production env and anything else “preview” deploys will be in the preview env. it works as you’d imagine.

just set a domain to a branch that’s not the master/default branch.

[mfferreira]

I believe you can opt in for beta here

http://zeit.co/_flags

Thank you!! 🥇
Enabling the _Environment Variables_ flag just solved it flawlessly!

Just spent the last 24h looking for how to set prod/preview env variables, and all the solutions are geared towards using now cli, which is a no-go when using now deployment hooks.

[Noste]

If you are using a monorepo, how can you define which env variables have to go with which apps ? It seems that it was possible before v2 of Now since you could specify env in builds but I don't get how it can be done in v2.

[LeoMartinDev]

I'm currently trying to use .env.production on a Vue cli project but my env files are not used when webpack build occurs :(

[mayatools]

Has the "Environment Variables" option been removed from http://zeit.co/_flags ? I no longer see it? Our project env't variables are geared around using .env files.

[pke]

Its now an offical project setting.

[styfle]

• Announcement: https://vercel.com/blog/environment-variables-ui
• Documentation: https://vercel.com/docs/v2/build-step#environment-variables
• CLI: vercel env --help