Git-secret: How do I use git-secret when deploying to ephemeral infrastructure?

[mrmachine]

For example, Docker Cloud. Inside the Docker image, there is no personal GPG secret key available to decrypt the secrets. All I can do to configure the image is define environment variables in a Docker Cloud stack file (YAML).

I am anticipating it will be a major PITA to ask people to export a secret key for deployment in ASCII so it can be passed through to the container via environment variable (and also the passphrase).

The only idea I have so far is to create a project/deployment specific GPG secret key and actually commit it to the Git repository and copy it into the Docker image, making sure that it is protected with a _very_ strong passphrase. Then passing just the passphrase through to the container via the Docker Cloud stack file at runtime.

For local development, individual developers would still use their own personal GPG secret keys to decrypt (if they need/have access). Only DevOps would know the passphrase to the project/deployment specific GPG secret key. It would necessarily be stored somewhere on Docker Cloud's configuration management infrastructure, but right now all the individual secrets are stored in a Docker Cloud stack file anyway.

Does that sound reasonable? Or do you have a better suggestion?

Btw, great tool. I compared a bunch of similar alternatives and git-secret seems to have the best balance with a clear and easy to use CLI and opaque behaviour.

[sobolevn]

Thanks for the good question!

Short intro, I don't really have that much experience with the cloud-deployment platforms, except a little bit of heroku and azure. But they have completely different setups from container-based infrastructures.

I agree with you about the individual keys, that's not the way to do it. As it compromises a lot of private data.

I also agree with you about the chosen solution. the only other possibility I see is to create a custom deployment plugin, which would ask for the key/pass without storing them. But that's surely not possible on every provider.

For now I still have some other questions to answer, like: jenkins workflow and CI in general (and CD as a part of the process). I guess I will write an article about it soon.

So I would highly appreciate any other feedback from you.

[mrmachine]

We use Travis CI to build, test and push (if tests pass) a Docker image. When Docker Cloud detects a new image, it can be configured to auto-redeploy (we enable this on staging, not production). So it needs to be completely automated. No interactively asking for key/passphrase.

We don't actually need to decrypt the secrets on Travis CI to run tests or build an image. Only on staging/production. The secrets contain credentials to staging/production database and email account, etc. which are not needed for tests to run.

Travis CI does also allow us to provide secret environment variables if needed (e.g. Docker Cloud credentials so we can push the resulting image, or a git-secret passphrase if those secrets were required to run tests).

I've written a bash script that we can run once in a new project to configure that project to use git-secret. E.g. generate a project/deployment specific GPG identity with a random password, run git secret init, and add the .gitsecret and .gnupg directories to the index ready to commit.

It can also be run in production environments as a wrapper around the actual web server or other command which needs access to the secrets. e.g. setup-git-secret.sh gunicorn ..., and will expect to find $GPG_PASSPHRASE already defined in the environment and will reveal any encrypted secrets before running the wrapped command (e.g. gunicorn ...)

Here's the script:

#!/bin/bash

# Initialise git-secret, generate a GPG encryption key, configure git-secret,
# decrypt all known secrets, and execute a command.

cat <<EOF
# `whoami`@`hostname`:$PWD$ setup-git-secret.sh $@
EOF

set -e

cd "$ICEKIT_PROJECT_DIR"

# Place GPG home directory in project directory.
export GNUPGHOME="$ICEKIT_PROJECT_DIR/.gnupg"

# Derive a fictitious email address for the encryption key.
export GPG_EMAIL="${GPG_EMAIL:-git-secret@$ICEKIT_PROJECT_NAME.icekit.interaction.net.au}"

# Generate a strong random passphrase.
export GPG_PASSPHRASE="${GPG_PASSPHRASE:-$(openssl rand -base64 48)}"

if [[ -d .git && ! -d .gitsecret ]]; then
    # Initialise git-secret.
    git secret init >/dev/null

    # Generate an encryption key for unattended decryption.
    gpg2 --batch --gen-key <<-EOF 2>/dev/null
        Key-Type: RSA
        Key-Usage: encrypt
        Passphrase: $GPG_PASSPHRASE
        Name-Email: $GPG_EMAIL
        EOF
    cat <<EOF
###############################################################################
# An encryption key has been generated for git-secret, with a random passphrase
# that will only be displayed one time, below:
#
#     $GPG_PASSPHRASE
#
# Make sure you add this passphrase to your password manager. You will need it
# to decrypt secrets in production environments.
#
# Quick start:
#
#     $ git secret add file  # Add 'file' as a secret to be encrypted
#     $ git secret hide      # Encrypt all secrets
#     $ git secret reveal    # Decrypt all secrets
#
# It is recommended to add 'git secret hide' to your pre-commit hook, so you
# won't miss any changes.
#
# For more information, see: http://sobolevn.github.io/git-secret/
###############################################################################
EOF

    # Configure git-secret.
    git secret tell "$GPG_EMAIL" >/dev/null

    # Add to index, ready to commit.
    git add .gitsecret
    git add .gnupg
fi

# List and decrypt all known secrets.
SECRETS="$(git secret list 2>/dev/null)"
if [[ -n "$SECRETS" ]]; then
    cat <<-EOF
        Secrets:
        $SECRETS
        Decrypting...
        EOF
    git secret reveal -p "$GPG_PASSPHRASE"
fi

# Execute command.
exec "$@"

I have couple more questions, though.

With the script above, it adds a pubring.gpg~ file to both the .gitsecret and .gnupg directories, and gpg.conf to .gnupg. When I run git secret hide, it also creates random_seed in the .gitsecret directory. Do I need to commit these to the repo, or can/should I add them to .gitignore?

Also, the Docker image might be mounted read-only for some deployments. Does git-secret or GPG need to write any files during decryption?

Finally, it would be great if you could include in your docs an example pre-receive hook to reject unexpected keys being added, OR even better would be to have git secret hide interactively list newly added keys (since last time secrets were hidden) and prompt you for confirmation. I can imagine that many people will not bother with the pre-receive hook, and even if one were implemented, that might make it more difficult to add legitimate keys, and without the hook it would be very easy for additional keys to be added without anyone realising.