Aws-codebuild-docker-images: Using local AWS credentials for Local CodeBuild

[jakebenn]

Is it possible to pass through local AWS credentials (e.g. from ~/.aws) to the build container when running CodeBuild locally? This would be ideal when making AWS CLI calls in buildspec.yml. Otherwise, it's not clear how to pass through an ACCESS_KEY_ID and SECRET_ACCESS_KEY to the build container.

[heitorlessa]

@subinataws just bumped this same issue here too - Tried mounting my ~/.aws as a Docker volume for the build environment or passing environment variables and they don't get passed through when the local agent runs the docker container using IMAGE_NAME.

This reads as a feature request here and would be very useful as most of the builds use AWS CLI - Happy to write a quick proposal

[omar-nahhas]

+1 This is needed to test anything that touches aws services

[cm-wada-yusuke]

+1
I found that environment variable can be passed using codebuild_build.sh, I tried it like this:
 
```env.txt
AWS_ACCESS_KEY_ID=xxx
AWS_SECRET_ACCESS_KEY=xxx
AWS_SESSION_TOKEN=xxx
AWS_DEFAULT_REGION=ap-northeast-1
AWS_DEFAULT_OUTPUT=json

./codebuild_build.sh
-i aws/codebuild/python:3.6.5
-a ./artifacts
-e ./env.txt

However, it did not work. In `buildspec.yml`, I simply set the environment variable from the parameter store, but the output is error status:

Attaching to agentresources_agent_1, agentresources_build_1
agent_1 | [Container] 2018/06/18 02:10:16 Waiting for agent ping
agent_1 | [Container] 2018/06/18 02:10:16 Waiting for DOWNLOAD_SOURCE
agent_1 | [Container] 2018/06/18 02:10:31 Phase is DOWNLOAD_SOURCE
agent_1 | [Container] 2018/06/18 02:10:31 CODEBUILD_SRC_DIR=/codebuild/output/src369896792/src
agent_1 | [Container] 2018/06/18 02:10:31 YAML location is /codebuild/output/src369896792/src/buildspec.yml
agent_1 | [Container] 2018/06/18 02:10:31 Processing environment variables
agent_1 | [Container] 2018/06/18 02:10:31 Decrypting parameter store environment variables
agent_1 | [Container] 2018/06/18 02:10:31 Phase complete: DOWNLOAD_SOURCE Success: false
agent_1 | [Container] 2018/06/18 02:10:31 Phase context status code: Decrypted Variables Error Message: MissingRegion: could not find region configuration
agent_1 | [Container] 2018/06/18 02:10:31 Runtime error (*awserr.baseError: MissingRegion: could not find region configuration)
agentresources_build_1 exited with code 0
agentresources_agent_1 exited with code 0
```

[Dlozitskiy]

As a workaround you can bake temporary security credentials into the build image every time they get expired:

temp_role=$(aws sts assume-role --role-arn arn:aws:iam::1234567890:role/codebuild-role--role-session-name "session_1" --profile your_profile_if_not_default)

echo AWS_ACCESS_KEY_ID=$(echo $temp_role | jq .Credentials.AccessKeyId)
echo AWS_SECRET_ACCESS_KEY=$(echo $temp_role | jq .Credentials.SecretAccessKey)
echo AWS_SESSION_TOKEN=$(echo $temp_role | jq .Credentials.SessionToken)

Add region to this list as well like:
AWS_DEFAULT_REGION="us-east-1"

Add those lines to Dockerfile:

$ cd ../aws-codebuild-docker-images/ubuntu/python/2.7.12/

$ tail -10 Dockerfile

ENV AWS_ACCESS_KEY_ID="***" \
    AWS_SECRET_ACCESS_KEY="***" \
    AWS_SESSION_TOKEN="***" \
    AWS_DEFAULT_REGION="***"

CMD ["python2"]

Build image:

$ docker build -t aws/codebuild/python:2.7.12 .
Run codebuild local referencing a new image:

docker run -i -v /var/run/docker.sock:/var/run/docker.sock -e "IMAGE_NAME=aws/codebuild/python:2.7.12" -e "ARTIFACTS=/artifacts" -e "SOURCE=/source/app-repo" amazon/aws-codebuild-local

[AWSstangers]

The latest release of the local agent solves this issue. Please download the latest image from dockerhub along with the updated wrapper script.

[lockenj]

@AWSstangers this doesn't seem to work for me...
I ran:

docker pull amazon/aws-codebuild-local:latest --disable-content-trust=false
docker build -t aws/codebuild/noaws/codebuild/nodejs:8.11.0
./codebuild_build.sh -i aws/codebuild/noaws/codebuild/nodejs:8.11.0 -a dist-codebuild -c
also tried
./codebuild_build.sh -i aws/codebuild/noaws/codebuild/nodejs:8.11.0 -a dist-codebuild -e codebuild.env
where codebuild.env had AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY and AWS_DEFAULT_REGION all set

Any info/help would be huge thanks!

[jakebenn]

I pulled the latest version from the GitHub repo, re-ran the steps in the AWS tutorial, and it's working for me now. I'm running:

./codebuild_build.sh -i aws/codebuild/python:3.6.5 -a ARTIFACT_DIR -s SOURCE_CODE_DIR -e ./.env -c

I'm not specifying AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY or AWS_DEFAULT_REGION in the .env file. Rather, the script is using the default AWS profile in ~/.aws.

One thing to check is your image identifier (the value after the -i flag). According the AWS documentation, the Node 8 image is aws/codebuild/nodejs:8.11.0. Not sure if that's an issue.

[lockenj]

@jakebenn I got further this time I am getting stuck on this error
Phase context status code: Decrypted Variables Error Message: parameter does not exist: coin-cobo-srfr-slack-incoming-web-hook

using aws ssm get-parameter --name coin-cobo-srfr-slack-incoming-web-hook --with-decryption from my local host machine it works fine. Any thoughts?