Chalice: Support encrypting sensitive config and per stage

[boosh]

I need to store sensitive data (e.g. passwords) for use by lambda. I'm currently planning on doing this by using a custom config file, but I see that Lambda supports KMS. Ideally I'm looking for a way that allows me to locally encrypt this data as well so I can safely store everything in my git repo (for example how ansible has a command to encrypt secrets using ansible vault) without adding extra tooling.

I plan on using separate accounts per stage to fully segregate my resources, so it'd useful if there was support for config-<stage>.json files as well. Doing that could allow these stage-specific config files to be encrypted separately per stage. It could require a password to decrypt the config for whichever stage while running/deploying.

Have you given any thought to how to handle encrypted config, and then push it up to lambda so it's encrypted there as well?

[jamesls]

The built in support for KMS with Lambda is to encrypt your env vars at rest. Lambda would automatically handle the decryption before invoking your function for you as described here.

We've discussed adding support for KMS in chalice so the process is more seamless, but it is possible to do this today, it's just more manual.

If you wanted the values encrypted client side you can follow a similar process to the "encryption helpers" that are in the AWS Lambda console. The process would be:

1. Create a KMS key if you don't already have one
2. Encrypt your variables via KMS:

$ aws kms encrypt --key-id $KEY_ID --plaintext secretpassword --output text --query CiphertextBlob
AQICAHjZ+JlI8KKmiVc++NhnBcO0xX3LFAaCfsjH8Yjig3Yr2AFPIyKCp3XVOEDlbiTMWiejAAAAbDBqBgkqhkiG9w0BBwagXTBbAgEAMFYGCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQMhai9vkA2KdU5gd+qAgEQgCnWW4F3fb7pTwmA2ppskJhUl0dJGEXIE5oDCr3ZsH7TlN5X381juPg0LA==

1. That encrypted value you'd store in your config file. You can have environment variables specified per stage:

{
  "version": "2.0",
  "app_name": "app",
  "stages": {
    "dev": {
      "environment_variables": {
        "SECRET_DATA": "AQICAHjZ+JlI8KKmiVc++NhnBcO0xX3LFAaCfsjH8Yjig3Yr2AFPIyKCp3XVOEDlbiTMWiejAAAAbDBqBgkqhkiG9w0BBwagXTBbAgEAMFYGCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQMhai9vkA2KdU5gd+qAgEQgCnWW4F3fb7pTwmA2ppskJhUl0dJGEXIE5oDCr3ZsH7TlN5X381juPg0LA=="
       }
    },
    "prod": {
      "environment_variables": {
        "SECRET_DATA": "other-encrypted-value-for-prod"
       }
      }
    }
  }
}

More info here: http://chalice.readthedocs.io/en/latest/topics/configfile.html#stage-specific-configuration

1. In your app code you'd decrypt the value when you need to (you'll have to ensure if you're IAM role has the ability to use the key for decryption:

>>> kms = boto3.client('kms')
>>> response = kms.decrypt(CiphertextBlob=os.environ['SECRET_DATA'].decode('base64'))
>>> secret_data = response['Plaintext']
>>> secret_data
'secretpassword'

Let me know if you have any more questions. Otherwise I'll mark this as a feature request. Others please feel free to +1 this issue if this is something you'd like chalice to help manage.

[boosh]

Thanks for your detailed response. I'll dig in and give it a try.

[AmineS]

Another alternative is to use dotenv and not rely on the config.json for environment variables.

[tmcfarlane]

I ran into this as well. Trying to see if I can use gitlabs protected environment variables to accomplish this. Will update if I find anything.

[tmcfarlane]

Went ahead with @jamesls approach here: https://github.com/aws/chalice/issues/481#issuecomment-323178711

Works well. The only downside is you have to add some decrypt logic in your code along with appropriate decoding.

My helper function ended up looking like this:

def decrypt_sec_key_with_kms_key(sec_key):
    # decrypt sec key 
    kms = boto3.client('kms')   
    response = kms.decrypt(CiphertextBlob=base64.b64decode(sec_key))    # decode key before decrypting it
    return response['Plaintext'].decode('utf-8')    # decrypted key comes in bytes literal form, must decode to utf-8

This was for python 3.6.5. Hope this helps someone!

[dmulter]

Would love to see this and related issues rolled into a new proposal. This is a key foundation piece any time you build a real Chalice app. A lot of us are implementing these pieces ourselves using various techniques. Seems like this issue is on the right track, but needs to be turned into a specific plan.