Aws-cdk: Cross-account pipeline doesn't generate sufficient permissions to deploy lambda in another account

[kamarouski]

:bug: Bug Report

What is the problem?

When using cross-account CloudFormation actions within pipeline to deploy Lambda the stack execution fails with the following error:

Your access has been denied by S3, please make sure your request credentials have permission to GetObject for ARTIFACTS_BUCKET_PATH

Error Code: AccessDenied. S3 Error Message: Access Denied (Service: AWSLambdaInternal; Status Code: 403; Error Code: AccessDeniedException;

This happening because account deployment role doesn't have permissions to both artifacts bucket and KMS key used by the pipeline. CDK only generates permissions for pipeline action role.

Reproduction Steps

1. Create a pipeline that deploys lambda into another account
2. Set lambda source code being from pipeline artifacts bucket
3. Deploy generated supporting stack to lambda account
4. Deploy pipeline stack to pipeline account
5. Execute the pipeline

Result: Pipeline fails due to insufficient permissions to deploy lambda.

Environment

• CDK CLI Version: 1.4.0
• Module Version: 1.4.0
• OS: OSX Mojave
• Language: TypeScript

[jpeddicord]

(Customer hat on -- I don't work on CDK!)

Can reproduce. The KMS key on the bucket artifact effectively prevents a CloudFormation pipeline action from starting (it can't even create a change set). Disabling KMS on the bucket and the individual object that's trying to be deployed will allow that to continue, but that's obviously not the right solution.

[skinny85]

Thanks @jpeddicord . The solution here is to add permissions to the deployment Role, not to disable encryption:

myPipeline.artifactBucket.grantRead(myCloudFormationAction.deploymentRole);

(this will include both the Bucket and the Key)

Thanks,
Adam

[jpeddicord]

@skinny85 Thanks; that makes sense, and adding that did indeed attach some additional permissions to my bucket policy and key. However, I'm not sure that's it (or that it's all of the problem); even after applying that I still observe 403s at action time.

Here's something probably relevant: the S3 bucket that the CDK configures for artifacts has the correct default encryption key. But when CodeBuild writes output objects to that bucket, they don't use the CDK's CMK. They use the "generic" AWS-managed aws/s3 key on the account. That I think might be what's ultimately preventing consuming stacks from accessing the artifacts. I'll tinker and see if I can get that to use the correct key.

[skinny85]

Yes, that is a separate problem :). A workaround can be to set the Project key to be the same as the Pipeline's Key:

new codebuild.PipelineProject(this, 'Project', {
  // ...
  encryptionKey: myPipeline.artifactBucket.encryptionKey,
});

That should make it work.

[jpeddicord]

Awesome, there it is. Thanks for your help. Both of these things feel like things the CDK should implicitly do; cross-account or not. But I don't think I have the full picture of how that should work. Would these be considered bugs in the CDK, or user/documentation?

[skinny85]

The first one is a miss in the CDK (hence this issue 😊). The second... the jury's still out on that one :).