Drone: Help Debug why secrets don't work

[bradrydzewski]

With 0.5 secrets are a bit more complex than prior versions and I propose we provide some utilities to help individuals understand how they work and debug. Sort of a like a sql explain plan, I propose a drone secret explain command.

The command could look like this:

$ drone secrets explain drone/drone
$ drone secrets explain --pull_request drone/drone # for pull request
$ drone secrets explain --pull_request --context=path/to/.drone.yml drone/drone

Example output could look like this:

$ drone secrets explain --pull_request drone/drone

secret: MY_SECRET
image "gynzy/docker-drone-node:*" does not match "gynzy/docker-drone-node"

secret: HEROKU_TOKEN
image "plugins/heroku:*" matches "plugins/heroku:latest"
event [tag, push] does not match pull_request

secret: AWS_KEY
image "plugins/s3" matches "plugins/s3"
event [pull_request, push] matches pull_request

It is also possible that we could run the explain plan on the Yaml parsing and compiling itself to show how it is transformed. This might be even more helpful but will be way more difficult and involved.

[appleboy]

Nice command. 👍

[bradrydzewski]

Until we have this command available to help locally debug how and when secrets are injected, hopefully this will help people debug issues with secrets not being injected.

Overview

This will discuss common reasons secrets are not injected or are not working as expected. The overwhelming majority of issues with secrets are user error, so please read carefully and double check your work.

Known Issues

There is only one known bug with secrets related to signing the yaml file on Windows machines. If you use Windows and would like to help resolve this issue please see https://github.com/drone/drone/issues/1662

Image mismatch

When you add a secret you need to provide a list docker image(s) that the secret can be exposed to. The list of images must be an exact match to what is in your yaml, or must use matching logic.

Let's use this yaml as an example:

pipeline:
  publish:
    image: plugins/docker

These command will expose secrets:

drone secret add --image plugins/docker DOCKER_USERNAME octocat
drone secret add --image plugins/* DOCKER_USERNAME octocat

These commands will not expose secrets:

drone secret add --image docker DOCKER_USERNAME octocat # missing plugins/ prefix
drone secret add --image publish DOCKER_USERNAME octocat # publish is the step name, not the image name

Note that if you are using tags in the image name, they need to be taken into account when you specify the --image flag

pipeline:
  publish:
    image: plugins/docker:latest

These commands will expose secrets

drone secret add --image plugins/docker:latest DOCKER_USERNAME octocat
drone secret add --image plugins/docker:* DOCKER_USERNAME octocat

Pull Requests

For security reasons Drone will not expose secrets for pull request events. If you want secrets exposed to pull requests it needs to be enabled when adding the secret:

drone secret add --event pull_request --image plugins/docker:latest ...

Invalid or Missing Signature

For security reasons, Drone will not expose secrets unless the yaml is signed and that signature matches the contents of the yaml. If the yaml file changes, the signature needs to be updated.

If you don't have a .drone.yml.sig file secrets will not work. Please read the full documentation which explains how to sign your yaml file.

If you have a .drone.yml.sig but the signature is invalid, you will see a warning message above your build logs. You will need to re-sign the yaml and push the updated signature file

Malformed Secrets from File

When loading secrets from file, such as ssh keys, we recommend using the@ symbol followed by the filepath to load secrets as shown below. Attempting to cat the secret generally results in malformed values.

drone secret add --image=plugins/ssh SSH_KEY @path/to/id_rsa

Injection is Deprecated

Please note that Drone 0.5 changes how secrets are provided to your build steps. You may no longer inject secrets using the $$VARIABLE syntax. This will not work:

pipeline:
  publish:
    image: plugins/docker
    username: $$DOCKER_USERNAME

Instead Drone 0.5 will pass your secrets (ie DOCKER_USERNAME) directly to the plugin or build container as an environment variable. Plugins define the environment variable names they expect, and they will look for the appropriate values at runtime:

pipeline:
  publish:
    image: plugins/docker
-   username: $$DOCKER_USERNAME

[naveensrinivasan]

Thanks, here is a sample file https://github.com/naveensrinivasan/drone-test/blob/master/.drone.yml

Here is the output for my global secrets

GuessWho➜  drone : master ✔ : ᐅ  k8s:(gke_revature-tools_us-west1-a_drone) drone global secret add --image 'busybox:*' --event pull_request --skip-verify   PASSWORD octocat
GuessWho➜  drone : master ✔ : ᐅ  k8s:(gke_revature-tools_us-west1-a_drone) drone global secret ls
PASSWORD
Images: busybox:*
Events: push, tag, deployment, pull_request
SkipVerify: true
Conceal: false

And I am still not getting output for $PASSWORD

Here is the output from the build

+ printenv
DRONE_BRANCH=master
DRONE_COMMIT_AUTHOR_AVATAR=https://avatars.githubusercontent.com/u/172697?v=3
DRONE_JOB_FINISHED=0
CI=drone
DRONE_VERSION=0.5.0+822
HOSTNAME=313f640c0d30
DRONE_REPO_LINK=https://github.com/naveensrinivasan/drone-test
DRONE_COMMIT_AUTHOR=naveensrinivasan
DRONE_PREV_BUILD_NUMBER=1
SHLVL=2
HOME=/root
DRONE_COMMIT_BRANCH=master
DRONE_REPO_PRIVATE=false
DRONE_REPO_SCM=git
DRONE_BUILD_STATUS=success
DRONE_ARCH=linux/amd64
DRONE_PREV_COMMIT_SHA=845273d06c7640a98f116790975416cd16e2d5a4
DRONE_COMMIT_MESSAGE=Testing Build
DRONE_REPO_BRANCH=master
DRONE_YAML_VERIFIED=false
DRONE_REMOTE_URL=https://github.com/naveensrinivasan/drone-test.git
DRONE_JOB_STATUS=running
DRONE_REPO_OWNER=naveensrinivasan
DRONE=true
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
DRONE_BUILD_NUMBER=2
DRONE_BUILD_STARTED=0
DRONE_REPO_TRUSTED=true
DRONE_BUILD_LINK=http://drone.revatureengineering.com/naveensrinivasan/drone-test/2
DRONE_YAML_SIGNED=false
DRONE_COMMIT=0290214e5cc2747ab99dde6ebfb3e1832723b439
SHELL=/bin/sh
DRONE_JOB_NUMBER=1
DRONE_JOB_STARTED=1482715517
DRONE_BUILD_EVENT=push
DRONE_REPO_NAME=drone-test
DRONE_BUILD_CREATED=1482715387
DRONE_COMMIT_SHA=0290214e5cc2747ab99dde6ebfb3e1832723b439
[email protected]
DRONE_JOB_EXIT_CODE=0
DRONE_COMMIT_REF=refs/heads/master
DRONE_REPO_AVATAR=https://avatars.githubusercontent.com/u/172697?v=3
DRONE_PREV_BUILD_STATUS=failure
PWD=/drone/src/github.com/naveensrinivasan/drone-test
DRONE_COMMIT_LINK=https://github.com/naveensrinivasan/drone-test/commit/0290214e5cc2747ab99dde6ebfb3e1832723b439
DRONE_BUILD_FINISHED=0
DRONE_REPO=naveensrinivasan/drone-test
+ echo $PASSWORD

And it is working when I pass the secret locally

drone exec -secret  PASSWORD=naveen
Running Matrix job #0
[build:L0:0s] + printenv
[build:L1:0s] DRONE_BRANCH=master
[build:L2:0s] DRONE_JOB_FINISHED=0
[build:L3:0s] DRONE_VERSION=0.5.0+dev
[build:L4:0s] CI=drone
[build:L5:0s] HOSTNAME=f8352083e7ad
[build] exit code 0
[build:L6:0s] DRONE_PREV_BUILD_NUMBER=0
[build:L7:0s] SHLVL=2
[build:L8:0s] DRONE_COMMIT_BRANCH=master
[build:L9:0s] HOME=/root
[build:L10:0s] DRONE_REPO_SCM=git
[build:L11:0s] DRONE_REPO_PRIVATE=false
[build:L12:0s] DRONE_BUILD_STATUS=success
[build:L13:0s] DRONE_ARCH=linux/amd64
[build:L14:0s] DRONE_YAML_VERIFIED=true
[build:L15:0s] DRONE_JOB_STATUS=running
[build:L16:0s] DRONE=true
[build:L17:0s] PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
[build:L18:0s] DRONE_BUILD_NUMBER=0
[build:L19:0s] DRONE_BUILD_STARTED=0
[build:L20:0s] DRONE_REPO_TRUSTED=false
[build:L21:0s] DRONE_BUILD_LINK=http://drone.revatureengineering.com///0
[build:L22:0s] DRONE_YAML_SIGNED=true
[build:L23:0s] SHELL=/bin/sh
[build:L24:0s] DRONE_JOB_STARTED=1482716492
[build:L25:0s] DRONE_JOB_NUMBER=0
[build:L26:0s] DRONE_BUILD_EVENT=push
[build:L27:0s] DRONE_JOB_EXIT_CODE=0
[build:L28:0s] DRONE_BUILD_CREATED=0
[build:L29:0s] DRONE_COMMIT_REF=refs/heads/master
[build:L30:0s] PWD=/drone/src
[build:L31:0s] DRONE_BUILD_FINISHED=0
[build:L32:0s] PASSWORD=naveen
[build:L33:0s] + echo $PASSWORD
[build:L34:0s] naveen

Please let me know how I could add secrets. I have tried following command and none of them have solved my issue

1. drone global secret add --image 'busybox:*' --event pull_request --skip-verify PASSWORD octocat
2. drone global secret add --image 'plugins/busybox:*' --event pull_request --skip-verify PASSWORD octocat
3. drone global secret add --image 'docker/busybox:*' --event pull_request --skip-verify PASSWORD octocat

[bradrydzewski]

Based on the CLI output, busybox:* will not match busybox in you yaml

[bradrydzewski]

closing since secret impl changes to interpolate using ${variable}. This, combined with the error message regarding lack of signature, should suffice.