Aws-cdk: Using Secrets Manager for GitHub token fails authentication during deploy because it is not resolved

[redi-kilicb]

Note: for support questions, please first reference our documentation, then use Stackoverflow. This repository's issues are intended for feature requests and bug reports.

• I'm submitting a ...

  • [X] :beetle: bug report
  • [ ] :rocket: feature request
  • [ ] :books: construct library gap
  • [ ] :phone: security issue or vulnerability => Please see policy
  • [ ] :question: support request => Please see note at the top of this template.

• What is the current behavior?
  If the current behavior is a :beetle:bug:beetle:: Please provide the steps to reproduce

Current Behavior:
When you use a secret you stored in Secrets Manager as the oauthToken, the CloudFormation template that gets generated only has a resolve placeholder for it during cdk deploy:
https://github.com/aws/aws-cdk/blob/master/packages/%40aws-cdk/aws-codepipeline-actions/lib/github/source-action.ts#L88

It tries to authenticate with GitHub using that instead of the actual decrypted value
(example:
"OAuthToken": "{{resolve:secretsmanager:arn:aws:secretsmanager:us-east-1:111:secret:my-secret:SecretString:::}}"

Steps to Repro:
1)Create a secret that you can access using the Secrets Manager. Note its arn.
2)Create a source action for pipeline using GitHubSourceAction where you get the oauthToken from the Secrets Manager:

const sourceAction = new codepipeline_actions.GitHubSourceAction({
      actionName: 'GitHub_Source',
      branch: 'your-branch',
      output: new codepipeline.Artifact(),
      owner: 'your-repo-owner',
      repo: 'your-repo-name',
      oauthToken: secretsmanager.Secret.fromSecretArn(this, 'your-token-name', 'your-token-arn').secretValue,
      trigger: codepipeline_actions.GitHubTrigger.WEBHOOK
    });

3) Use this action in your pipeline:

new codepipeline.Pipeline(this, 'your-pipeline-name', {
      stages: [
        {
          stageName: 'Source',
          actions: [
            sourceAction
          ],
        },
        //...
      ]});

4) Build and try to deploy

• What is the expected behavior (or behavior of feature suggested)?

You should handle the case where the token is not entered using plain text (when using plain text, there are no issues because the generated CloudFormation template has the plain text token in there instead of "{{resolve:")

This is the plain text usage that works:

oauthToken: cdk.SecretValue.plainText('your-plain-text-token')

You could allow us to set registerWithThirdParty to false when instantiating a GitHubSourceAction when the token is a secret that's not created from plain text. Currently there is no way to set this flag without changing the generated CloudFormation template and then upload that, instead of using cdk deploy directly.

• What is the motivation / use case for changing the behavior or adding this feature?
  So you can use the CDK for GitHub as your source.

• Please tell us about your environment:

  • CDK CLI Version: 1.2.0
  • OS: macOS Mojave
  • Language: TypeScript

• Other information (e.g. detailed explanation, stacktraces, related issues, suggestions how to fix, links for us to have context, eg. associated pull-request, stackoverflow, gitter, etc)

[SanderKnape]

What works for me is the following:

const gitHubOAuthToken = SecretValue.secretsManager('platform/github/oAuthToken', {
    jsonField: 'oAuthToken',
});

const sourceAction = new GitHubSourceAction({
    oauthToken: gitHubOAuthToken,
    [...]
});

I'm not sure if what you are trying to do is supposed to work, but this at least will allow you to continue!

[redi-kilicb]

@SanderKnape thanks for your reply but turned out the issue is the way webhooks are activated.

Basically, if it's the first time you're connecting to your GitHub repo using a personal access token, you can't authenticate through cdk deploy. I had to generate the CloudFormation template using cdk synth, then manually override the RegisterWithThirdParty to false in the template, upload that template through the AWS Console, and then go into the pipeline (which fails the first time), edit the Source stage manually in the console, and click Connect to Github. Once I've done that, I can use cdk deploy with the RegisterWithThirdParty set to true from then on.

I closed the issue because it's not the CDK - it's CloudFormation that's unable to get the webhook working without manual intervention.

[skinny85]

You don't have to go through that whole dance @redi-kilicb . You can just go to the CodePipeline AWS Console, and click "Connect to GitHub" while creating a new Pipeline with a GitHub source (you don't even need to finish creating it, after the connection is established you can safely abort it).

You need to do this once per each AWS region you want to create Pipelines in.

[redi-kilicb]

An update in case this helps someone else: my assumption in my comment above did not turn out to be correct.

After I spent time chatting with AWS Support and trying to upload the template using different formats for the secret, it turns out that if you don't specify the jsonField like @SanderKnape has above in his example, the generated template value is incorrect. So thank you @SanderKnape for that!

When you don't specify jsonField, the generated value in the CloudFormation template looks like this:
"OAuthToken": "{{resolve:secretsmanager:my-token-name:SecretString:::}}"
When you do, it looks like this:
"OAuthToken": "{{resolve:secretsmanager:my-token-name:SecretString:my-token-name::}}"

I think having the options (SecretsManagerSecretOptions) as optional in secretsManager is misleading because it doesn't work without it.

[redi-kilicb]

You don't have to go through that whole dance @redi-kilicb . You can just go to the CodePipeline AWS Console, and click "Connect to GitHub" while creating a new Pipeline with a GitHub source (you don't even need to finish creating it, after the connection is established you can safely abort it).

You need to do this once per each AWS region you want to create Pipelines in.

@skinny85 We can't do that with my organization because the enterprise GitHub repo access is controlled by an internal group and they deny those requests you submit through the Console. The only way to connect to GitHub is to provide a Personal Access Token (even SSH keys are disabled in my org). And I haven't found a way to provide a Personal Access Token through the AWS Console when you try to create a Pipeline. If you have, let me know.

[skinny85]

OK. I think there's some confusion.

You need the jsonField property if the value you stored in in Secrets Manger is a key-value pair. Like this:

{
  "my-token-name": "my-secret-token"
}

However, you can also store a plaintext value under a given secret name. Like this:

In that case, you should _not_ provide the jsonField property.

For the Token: you need to provide an OAuth token for an account that has given the CodePipeline app in the given AWS region permissions to access that account. You can check that in Settings -> Applications -> Autohorized OAuth Apps in your GitHub account.

[swyp-de]

I am at the same stage currently and regarding plaintext secret. So its definitely ok to store the github token in the plaintext field of the SecretManager console. The textfield where the default is

{
  "": ""
}

So just removing the map json structure and put just the token in there?

[logemann]

To answer my own question: Yes, just remove the struct and put in the token.

[skinny85]

Yes, exactly. The default in the textfield is this:

{
  "": ""
}

You can instead put your token directly in there:

No quotes, no JSON, just the character contents of your token.