Dpl: Provider Lambda - Environment Variables in deployment

[DaMaillard]

Now, AWS Lambda functions allow environment variables.
It would be interesting to send it in the Travis deployment

http://docs.aws.amazon.com/fr_fr/lambda/latest/dg/env_variables.html

[mboudreau]

Can someone from Travis please comment on this as we need this and the ability to create secure environment variable sooner rather than later

[mboudreau]

@DaMaillard tried my hand at it. Easy enough to change the DPL code to support it, but because of Travis' horrible decision to limit the variables available to the provider when parsing the yml file I can't add a new variable to the .travis.yml file to include environment variables, and no way to test this end to end. I believe that the travis-yml project is included with the travis build and there's no way to modify this easily.

Only way to go around this is to use one of the 'approved' variables and create a JSON string of some kind, then parse it and add it to the environment variable when uploading the lambda function. Can't say I'm a fan.

[DaMaillard]

Good job @mboudreau
My current way of doing is also frustrating, but is different : I wrote a script bash that send it after the deployment. :

deploy:
  - provider: lambda
    function_name: "lambda_A"
    [...]
  - provider: script
    script: scripts_lambda_A.sh 

This scripts_lambda_A.sh uses the AWS CLI (installed by travis in the part - install) to send the variables, it has also the benefit to enable the encryption of some variables.

(And by the way, I also use this script to set up the triggers of my lambda, also missing in travis !)

It works well, in the meantime, but it would be great that travis includes it.

[iamkirkbater]

+1 on this issue. Would love to be able to deploy with environment variables, even if they're just a reference to a JSON file like with the AWS SDK.

Or at least have the deploy keep the existing environment variables and not nuke them all.

[baynezy]

+1
This really is a must have to do anything that involves a Lambda function working with anything that isn't an AWS service, as I don't want to hard-code credentials into my open source.

[baynezy]

@DaMaillard - do you have a complete example of the work around that you posted anywhere?

[mboudreau]

@baynezy I'm currently working on a PR to fix all the things that travis is missing out for Lambda deploy: https://github.com/travis-ci/dpl/pull/676

It's not ready yet, currently testing it out further since the tests for travis are very minimal and kinda useless.

[mboudreau]

Oh, and I forgot, still need to add the new keys to Travis-YML cause apparently, Travis wants to make everything overly complicated

[mboudreau]

There, another PR to fix travis-yml to recognize the new options available for lambda deployments: https://github.com/travis-ci/travis-yml/pull/7

[DaMaillard]

Great job @mboudreau. Hope it will be in prod quickly !

@baynezy :
Here is a simplified version of my script :
.travis.yml

[....]

deploy:
  - provider: lambda
    region: eu-west-1
    function_name: "my-lambda-production"
    role: "xxxxx"
    runtime: nodejs4.3
    access_key_id: $AWS_ACCESS_KEY_ID
    secret_access_key: $AWS_SECRET_ACCESS_KEY
    zip: "lambda_functions/xxxxx"
    module_name: "index"
    handler_name: "handler"
    timeout: 20
    memory_size: 1024
    skip_cleanup: true
    on:
      branch: master

  - provider: script
    script: python set-env-lambda.sh --branch=$TRAVIS_BRANCH
    skip_cleanup: true
    on:
      branch: master
[....]

set-env-lambda.sh :

#!/bin/bash
set -e
###############################################################################
# Set variables to a lambda environment 
###############################################################################

# Check the AWS Cli presence
which aws > /dev/null 2>&1

if [ $? != 0 ]; then
    echo "This utility requires the AWS Cli - You can add add 'pip install -r awscli' to the 'install' part of travis"
    exit -2
fi



if [ $# != 1 ]; then
    echo " ===  Please provide the branch of the lambda to trigger"
    exit -1
else
    BRANCH=$1
    REGION="eu-west-1"
fi

if [ -z ${VAR1+x} ]; then
    echo "VAR1 missing in Travis environment variables"
    exit -3
fi

echo "Branch $BRANCH"

if [ $BRANCH == "master" ]; then
  FUNCTION_NAME_LAMBDA="my-lambda-production"
  ARN_KEY="arn:aws:kms:eu-west-1:0123456789:key/1234-5678-4321-8765"
  VAR2="foo"

elif [ $BRANCH == "develop" ]; then
  FUNCTION_NAME_LAMBDA="my-lambda-staging"
  ARN_KEY="arn:aws:kms:eu-west-1:0123456789:key/1234-5678-4321-8765"
  VAR2="bar"
fi

echo " ===  Encryption of variable VAR1 "
VAR1_ENCRYPT=$(aws kms encrypt --key-id  "$ARN_KEY" --region "$REGION" --plaintext "$VAR1" | jq -c '.CiphertextBlob')

VARIABLES="{VAR1=$VAR1_ENCRYPT,VAR2=$VAR2}"


echo " ===  Setting $FUNCTION_NAME_LAMBDA Environment Variables "
aws lambda update-function-configuration \
    --region "$REGION" \
    --function-name "$FUNCTION_NAME_LAMBDA" \
    --environment Variables=$VARIABLES

But to be honest, this is a quick and dirty job, and will not be convenient with an increasing number of lambda or environments.

[tdmalone]

The docs now include an option for environment variables. Has this issue since been solved?

[stale[bot]]

Thanks for contributing to this issue. As it has been 90 days since the last activity, we are automatically closing the issue. This is often because the request was already solved in some way and it just wasn't updated or it's no longer applicable. If that's not the case, please do feel free to either reopen this issue or open a new one. We'll gladly take a look again! You can read more here: https://blog.travis-ci.com/2018-03-09-closing-old-issues

[stale[bot]]

Thanks for contributing to this issue. As it has been 90 days since the last activity, we are automatically closing the issue. This is often because the request was already solved in some way and it just wasn't updated or it's no longer applicable. If that's not the case, please do feel free to either reopen this issue or open a new one. We'll gladly take a look again! You can read more here: https://blog.travis-ci.com/2018-03-09-closing-old-issues