Chalice: Adding support for Systems Manager Parameter Store

[budowski]

Not 100% sure if you consider this under the scope of the project, but I think this would be extremely useful:
Adding support for Systems Manager Parameter Store (retrieving / setting params), so you could easily use more sensitive data, like API keys, credentials, etc., that you wouldn't want committed into your source code.
There is a nice implementation in this AWS blog post that could probably be integrated into Chalice:
https://aws.amazon.com/blogs/compute/sharing-secrets-with-aws-lambda-using-aws-systems-manager-parameter-store/

[jamesls]

There was a PR for it a while back that didn't get much traction: https://github.com/aws/chalice/pull/336

Would you want chalice to also manage the configuration of the SSM params (i.e creating the SSM params like the original PR did), or would you just want a convenient interface to retrieving SSM params (e.g app.get_param('some_param') which could translate the request to an ssm.get_parameter() call for/appname/stagename/some_param)?

[budowski]

I think a convenient interface for setting and retrieving SSM params would be really nice. Especially since it goes well with Lambda development

[switch180]

@budowski
What's the use case to set parameters from within the app? Couldn't a developer write a helper python script on the side to push new configurations to SSM's Parameter Store instead, or use the AWS console? I'm not declaring what the right pattern is here, but if you have a use case that would help. I really like this feature request!

I'd like it if we followed the a similar read-only model as Spring Cloud AWS, which uses paths separated by forward slashes. Introduced in spring-cloud-aws PR #308

The feature could:

• Load parameters from SSM parameter store (duh)
• Auto-populate environment variables from parameter store
• Allow defaults for each parameter to be set it the config.json, but not require each parameter be explicitly listed in order to be accessed at runtime
• Allow the parameters to be read in app.py during runtime, as shown in #336

Sample config.json
This sample config.json shows what the user might see. It adds a new config option called application_variables. To load a variable from ssm, the syntax might be "${ssm:/com/aws/switch180/my_app/db/prod}". Each variable has a name, which is important. It would allow you to 1) reference the value in other places in the config such as an environment variable and 2) set a default to take effect when/if network access is not available

[budowski]

Thanks for looking into this!
I think that this is more as a read-only access, which is the common case for using Chalice - so accessing sensitive information (like API keys, connection strings, etc.) will be done seamlessly through Chalice. I totally agree that setting these values shouldn't be part of the scope of the project, but more of a DevOps issue.
And having those SSM variables mapped in the config file is a good idea, as you can do different mappings according to deployment stages.

[brylie]

We are also evaluating Chalice for some internal projects, and use SSM to store/access secrets across our infrastructure. So, SSM support would be really important to allow us to securely manage read-only access to sensitive data like API keys, db authentication, etc.

Our Systems Coordinator notes that Boto3 and Serverless use a similar syntax:

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ssm.html#SSM.Client.get_parameter

[tedivm]

Sounds like a config class, similar to what Flask uses, would be great place to put this.

[Sytten]

Any chance we can get that feature?

[cb17]

+1

[jordan-dimov]

Any updates on this? Specifically read-only support for SSM parameters as an alternative to environment variables would be fantastic.

[Sytten]

It's not very hard to implement, I wrote a custom loader for our stack. You just need to deploy your role elsewhere and give it access to SSM.

[ricky-sb]

It's not very hard to implement, I wrote a custom loader for our stack. You just need to deploy your role elsewhere and give it access to SSM.

Would you mind sharing?

[Sytten]

Assuming you have your IAM correctly set (I created a role in terraform and used it in the chalice config).
I store my config as a single value in JSON in the parameter store, so your situation might be different.

# Adapted from the Flask configuration
# See the original at https://github.com/pallets/flask/blob/31d3c7e71941c3a1800f5e31064c9862da473d86/flask/config.py
class Config(dict):
    def __init__():
        dict.__init__(self, {})

    def from_mapping(self, *mapping, **kwargs) -> bool:
        mappings = []
        if len(mapping) == 1:
            if hasattr(mapping[0], "items"):
                mappings.append(mapping[0].items())
            else:
                mappings.append(mapping[0])
        elif len(mapping) > 1:
            raise TypeError(
                "expected at most 1 positional argument, got %d" % len(mapping)
            )
        mappings.append(kwargs.items())
        for mapping in mappings:
            for (key, value) in mapping:
                if key.isupper():
                    self[key] = value
        return True

def get_secret(name: str) -> Dict[str, str]:
    parameter_store = boto3.client("ssm")
    return parameter_store.get_parameter(Name=name, WithDecryption=True)["Parameter"]

def load_config(config_secret: str) -> Config:
    config = Config()
    data = json.loads(get_secret(config_secret)["Value"])
    config.from_mapping(data)
    return config

[skinofstars]

This is pretty convoluted compared to Serverless 😔

environment:
  SUPER_SECRET: ${ssm:/path/to/secureparam~true}

https://www.serverless.com/framework/docs/providers/aws/guide/variables#reference-variables-using-the-ssm-parameter-store