Aws-codebuild-docker-images: [aws/codebuild/standard:2.0] fails to load env.secrets-manager.key value defined in buildspec.yml (works on hosted codebuild, fails on local codebuild)

[the-vampiire]

i am trying to debug a problem with the build that is based on using a secret stored in the secrets manager.

the exact same buildspec.yml used on codebuild does load it properly, my subsequent usages of it are what i was trying to debug

• what i expected to happen: the local build will load the secrets manager variable the same as it does on the hosted codebuild

• what is happening: does not load the secrets manager variable defined by

env:
  secrets-manager:
    SECRET: <secret id>:<key name>

• what i have tried
  
  
  
  • works: manually using the CLI to load the variable (rules out account / permission / connectivity issue)
  
  
  • fails: using or echoing the variable
  
  
  • fails: with or without env.exported-variables
  
  

system info

• os: osx
• docker version: 19.03.5, build 633a0ea
• image used: aws/codebuild/standard:2.0

here is a short reproducible example

version: 0.2

env:
  exported-variables:
    # tried with and without using exported variables and still didnt work
    - SECRET
  secrets-manager:
    SECRET: <secret id>:<key name>

phases:
  pre_build:
    commands:
      # this works
      - aws secretsmanager get-secret-value --secret-id <secret id>
      # this does not
      - echo $SECRET
  install:
    runtime-versions:
      java: openjdk8

[awszhen]

@the-vampiire , thanks for reporting this issue. We will fix this issue in the next release.

[the-vampiire]

cool man. since your a literal codebuild dev would you happen to know if this is possible?

env:
  parameter-store:
    ENV_PARAM_NAME: /$ENVIRONMENT/param-name
  secrets-manager:
    ENV_SECRET_NAME: $ENVIRONMENT/secret-name:SECRET_KEY

where i could externally set $ENVIRONMENT to say dev or prod?

the use case is a buildspec that is identical for both dev and prod (with the exception of the parameter / secret name prefixes) so id like to parameterize them in a single source of truth. i havent been able to find any info on this online

[subinataws]

@the-vampiire - No, this isn't currently supported. It's on our backlog and should be addressed in an upcoming release. My best guess would be ~Q1 2020.

[the-vampiire]

hah 2 days of searching and trying and the expert gives me closure in 20 mins. thanks man

[subinataws]

@the-vampiire - This feature is now supported.

[the-vampiire]

cheers man thanks! 🍻

[yahel2410]

Hi @subinataws, I'm facing the same issue where secrets are not retrieved from secrets manager during local codebuild.
I cloned and built aws/codebuild/standard:2.0 image just yesterday, so it should include this fix right?

This is the cmd I run:
./codebuild_build.sh -c -i aws/codebuild/standard:2.0 -a /tmp/codebuild

My buildspec.yml:

version: 0.2
env:
  secrets-manager:
    USERNAME: <secretname>:username
phases:
  install:
    runtime-versions:
      python: 3.7
    commands:
      - echo $USERNAME  #prints empty line
...

[awszhen]

@yahel2410 , please use latest version of the local agent. SHA: f261c23999aa377fe9de61a9a35234b5af1d4b747b5d946549f581c70457427c

[yahel2410]

I can confirm it works with the latest image. Thank you

[celicoo]

@subinataws where can I set $ENVIRONMENT now that this is supported?

[awszhen]

@celicoo , you can set and use $ENVIRONMENT by using this buildspec:

env:
  variables:
    ENVIRONMENT: "value"
  parameter-store:
    ENV_PARAM_NAME: /$ENVIRONMENT/param-name
  secrets-manager:
    ENV_SECRET_NAME: $ENVIRONMENT/secret-name:SECRET_KEY