Nodebestpractices: Environment variables -- Secure access for CI-CD

[karthiknvlr]

Hi All,

Most of us are using Github for collaboration and code review.
In real time, most of the deployments are happening through dockers and CI-CD using Jenkins and other sophisticated tools.

So, In this way how to pass the environmental files for CI-CD? We can't hold the environmental variables in Github since every developer who has access can able to see it.

What is the best way to inject the environmental variables for building a docker image in CI-CD?

[goldbergyoni]

@karthiknvlr Welcome aboard 🌞

I find this a very challenging question that many struggles with. In fact, one of my customers recently quit circleci in favor of AWS just for the sake of unified security. I'll share my thoughts soon.

Looping also @lirantal and @itaysk who might as usual have some insights to share

[itaysk]

We can't hold the environmental variables in Github
What is the best way to inject the environmental variables for building a docker image in CI-CD?

Can you identify the motivation for the source/config separation? Is it because you (1) want to be able to make configuration pluggable (e.g config per environment) or because you (2) don't want to expose the content of the configurations (i.e secrets).

If it doesn't make sense to add the configuration to the repo (1), then the solution is based on your CI/CD tool of choice. Most of them offer a method to inject environment variables to the job. I also saw pipelines that start by fetching the config in runtime but I doubt this is necessary in your case. For plain Docker, there's -e and --env-file flags.

If these configurations are secret (2), then there are a couple of options: Some CI/CD tools have a native method to store and inject secret values. If your's doesn't, or you don't trust it I'd recommend looking into Mozilla SOPS (https://github.com/mozilla/sops). It allows you to encrypt the values before checking them into the repo. This way they are available for your CI/CD but no one can use them besides you and your CI/CD (or whoever has the key). I have used SOPS and successfully integrated it into a CD pipeline.
Other alternatives like Hashicorp Vault (https://www.vaultproject.io) offers a comprehensive solution but may be an overkill for your needs.

[karthiknvlr]

Hi @i0natan and @itaysk, Thanks for your inputs.

@itaysk My motive is to secure the secrets/credentials.

Let us take an example, In my current release, I have to add a feature to send a confirmation email during the registrations process. To make it possible I need to integrate one SMTP server or third party API's like SendGrid, SES etc. In this scenario, the developer will develop the feature and he will check into the Github. Once he pushed the code the CI-CD will start and it will deploy the container in development server.
Here, in the current system, we are checking in environmental variables (.env) also to GitHub along with the code. So that it can be used to build the docker.

Now, my questions are how to do it without checking in to the GitHub? In run time while creating the image it needs to be injected from other sources.

So that the production and staging credentails are known only to concern stakeholders(Product Owner).

[goldbergyoni]

@karthiknvlr I'd like to verify your question before suggesting my view: Your goal is making a secret available to your production run-time, correct? Also, seems like you hope to achieve this by baking the secret into the deployment container, right?

@itaysk In case my assumption above is correct, do you see any value in including the production secrets within the repo? putting aside the security concern, it seems just odd that a developer specifies the production secret while shipping a build. Wouldn't the native and convenient flow be that ops/developer during deployment reveal the secrets and inject into production (e.g. env variable UI, KMS, etc)?

[itaysk]

depends on the nature of the repo. If it's the actual source code, then you're probably right, but ft it's the helm chart that deploys the code, then it might be beneficial to include all config/secrets in there. And if you practice GitOps this becomes more obvious.

[goldbergyoni]

@karthiknvlr Sorry for the late reply. To my understanding you seek a method to make secrets available to your production and the CI/CD is where you aim to this.

I guess that in many no highly secured environments baking the secrets into the container within the CI/CD is reasonable, and as @itaysk suggested many vendors will allow you to define this within the job definition.

That said, I tend to believe it's not secured enough in many other circumstances because you bake all your secrets into a single artifact (Docker image) that is not traceable or expirable - anyone who will have access to a dev artifact (Docker image) will be exposed to production secrets. For example, your QA env which is exposed to the Docker image will have access to all secrets. Other and presumably better approach would be injecting the secrets directly into your env runtime (e.g. Kubernetes secrets, other platforms have UI for env var) or even better use a KMS that rotate these secrets, track their usage and enforce expiration

[karthiknvlr]

@i0natan Thanks for your suggestions. In our existing project, we baking it during the CD process. I will try to inject it through KMS or kubernetes secrets.

Could you please add CI-CD in Node best practices pipeline? So, it will be helpful for new bibes to move in the right path.

[goldbergyoni]

Sure thing, doing this soon