Aws-cdk: [cdk-pipelines] Every asset update mutates the pipeline

Created on 15 Jul 2020  Â·  11Comments  Â·  Source: aws/aws-cdk


The UpdatePipepline stage fails although the SelfMutate action has completed successfully and causes the pipeline to start over again. This leads to the pipeline getting caught in an infinite loop.

Reproduction Steps


I have a CDK app with a PipelineStack containing a CdkPipeline (with two Cdk.Stages). I committed my CDK code to git, and then run cdk deploy PipelineStack --profile ... (all environments were bootstrapped before).

Error Log


After creating the pipeline, the first pipeline execution is being triggered by a CreatePipeline event. Source and Build stage succeed, the SelfMutate action succeeds, but the UpdatePipeline stage fails. The next execution gets triggered by a StartPipelineExecution event (probably triggered by the pipeline itself?).

Screenshot 2020-07-15 at 14 43 38

I don't know where to find more detailed error messages of what causes the stage to fail. Please point me to the releavnt locations and I will provide more information.

Environment

  • CLI Version : 1.51.0 (build 8c2d53c)
  • Framework Version: 1.51.0 (build 8c2d53c)
  • Node.js Version: v12.18.0
  • OS : aws/codebuild/standard:4.0
  • Language (Version): TypeScript (3.9.5)

Other


Other users have reported the same issue on Gitter, but no resolution has been provided so far.

This is :bug: Bug Report

@aws-cdpipelines bug p1
Source
picture asterikx
👍6

Most helpful comment

Would it be possible to generate a token from the source path and use that as name for the publishing action, e.g. path.join(__dirname, '../web/dist') --> TOKEN 123 --> PublishAssets123. So hash changes would not require pipeline mutations. Though I'm not sure how the publish action would know which asset it should deploy in this case ...

If we only use the path to calculate the hash, the asset won't be invalidated when it changes and will never be updated.

As I mentioned, I think we need to figure out a way to avoid pipeline updates when assets are changed and the way to do that would probably be to define publishing actions that are not coupled directly with the asset hash but rather with the number of assets in the app. If a new asset is added, the pipeline will be updated (another publish action is needed) but if assets are only updated, the pipeline won't need to change because we will already have a publishing action for each asset, and only the ones that changes will actually upload data (the rest will succeed with no-op).

picture eladb on 21 Jul 2020
👍3

All 11 comments

I concur, this happened to me as well.

ferdingler picture ferdingler on 15 Jul 2020

I think a succeeded 'SelfMutate' action but a failed 'UpdatePipeline' stage is expected in case 'SelfMutate' did actually update the pipeline.

The pipeline has restartExecutionOnUpdate set to true, which makes perfect sense as the pipeline needs to be updated before proceeding.

https://github.com/aws/aws-cdk/blob/b0f8729002b90c1c90ca46a4db9e297a69fef174/packages/%40aws-cdk/pipelines/lib/pipeline.ts#L76

The wording "failed" and the visualization in the Console is confusing at first. But I think that's how CodePipeline deals with restarts internally, i.e. "if I need to start over, I have to fail the current execution so that the new execution can go through without being blocked by the old execution" (pipeline executions cannot overtake each other).

Looking at the logs of the CodeBuild project of the 'SelfUpdate' action, I see that my Pipeline is indeed being updated in each iteration, which explains the infinite looping.

Those updates to my Pipeline are asset updates (and associated role and policy updates).


Complete CodeBuild logs

[Container] 2020/07/15 12:57:16 Running command cdk -a . deploy PipelineStack --require-approval=never --verboseCDK toolkit version: 1.51.0 (build 8c2d53c)Command line arguments: { _: [ 'deploy' ],  a: '.',  app: '.',  'require-approval': 'never',  requireApproval: 'never',  verbose: 1,  v: 1,  'ignore-errors': false,  ignoreErrors: false,  json: false,  j: false,  ec2creds: undefined,  i: undefined,  'version-reporting': undefined,  versionReporting: undefined,  'path-metadata': true,  pathMetadata: true,  'asset-metadata': true,  assetMetadata: true,  'role-arn': undefined,  r: undefined,  roleArn: undefined,  staging: true,  'no-color': false,  noColor: false,  fail: false,  'build-exclude': [],  E: [],  buildExclude: [],  ci: false,  execute: true,  force: false,  f: false,  parameters: [ {} ],  'previous-parameters': true,  previousParameters: true,  '$0': '/usr/local/bin/cdk',  STACKS: [ 'PipelineStack' ],  stacks: [ 'PipelineStack' ] }merged settings: { versionReporting: true,  pathMetadata: true,  output: 'cdk.out',  app: '.',  context: {},  tags: [],  assetMetadata: true,  requireApproval: 'never',  toolkitBucket: {},  staging: true }Toolkit stack: CDKToolkitSetting "CDK_DEFAULT_REGION" environment variable to eu-central-1Resolving default credentialsLooking up default account ID from STSDefault account ID: 1111111111Setting "CDK_DEFAULT_ACCOUNT" environment variable to 1111111111context: { 'aws:cdk:enable-path-metadata': true,  'aws:cdk:enable-asset-metadata': true }--app points to a cloud assembly, so we bypass synthPipelineStack: deploying...Assuming role 'arn:aws:iam::1111111111:role/hnb659fds-deploy-role-1111111111-eu-central-1'.Waiting for stack CDKToolkit to finish creating or updating...[0%] start: Publishing 65a6bd9bc7f71e9fd2a17c7f41bf3d87f620c2930b4ed93db84329371c27:1111111111-eu-central-1Retrieved account ID 1111111111 from disk cacheAssuming role 'arn:aws:iam::1111111111:role/hnb659fds-file-publishing-role-1111111111-eu-central-1'.[0%] check: Check s3://cdk-hnb659fds-assets-1111111111-eu-central-1/65a6bd9bc7f71e9fd2a17c7f41bf3d87f620c2930b4ed93db84329371c27[0%] upload: Upload s3://cdk-hnb659fds-assets-1111111111-eu-central-1/65a6bd9bc7f71e9fd2a17c7f41bf3d87f620c2930b4ed93db84329371c27[100%] success: Published 65a6bd9bc7f71e9fd2a17c7f41bf3d87f620c2930b4ed93db84329371c27:1111111111-eu-central-1PipelineStack: checking if we can skip deployPipelineStack: template has changedPipelineStack: deploying...Attempting to create ChangeSet CDK-5bd31a53-9245-454a-b8cf-60f8fc3afcb6 to update stack lineStackPipelineStack: creating CloudFormation changeset...
Initiated creation of changeset: aws:cloudformation:eu-central-1:1111111111:changeSet/5bd31a53-9245-454a-b8cf-60f8fc3afcb6/e2a256d5-c1d6-46b3-a202-9e9c9b106672; waiting for o finish creating...
Waiting for changeset CDK-5bd31a53-9245-454a-b8cf-60f8fc3afcb6 on stack PipelineStack to sh creating...
Changeset CDK-5bd31a53-9245-454a-b8cf-60f8fc3afcb6 on stack PipelineStack is still ting
Changeset CDK-5bd31a53-9245-454a-b8cf-60f8fc3afcb6 on stack PipelineStack is still ting
Changeset CDK-5bd31a53-9245-454a-b8cf-60f8fc3afcb6 on stack PipelineStack is still ting
Changeset CDK-5bd31a53-9245-454a-b8cf-60f8fc3afcb6 on stack PipelineStack is still ting
Initiating execution of changeset CDK-5bd31a53-9245-454a-b8cf-60f8fc3afcb6 on stack lineStack
Execution of changeset CDK-5bd31a53-9245-454a-b8cf-60f8fc3afcb6 on stack PipelineStack started; waiting for the update to complete...
Waiting for stack PipelineStack to finish creating or updating...
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS (User Initiated))
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
  3/33 | 12:58:11 PM | CREATE_IN_PROGRESS   | AWS::IAM::Role              | Pipeline/line/Assets/3f399ce66b41b1c428a01d8b7eeaa7a3d41331c4b14d39325e6708e7d37836f0/PipelineActionRole elineAssets3f399ce66b41b1c428a01d8b7eeaa7a3d41331c4b14d39325e6708e7d37836f0CodePipelineAcRole9984FB3F) 
  3/33 | 12:58:12 PM | CREATE_IN_PROGRESS   | AWS::IAM::Role              | Pipeline/line/Assets/e3c385af9f0538d61e567e307305627f27b709bbe58c8efce25debed54d3ab21/PipelineActionRole elineAssetse3c385af9f0538d61e567e307305627f27b709bbe58c8efce25debed54d3ab21CodePipelineAcRole125C1262) 
  3/33 | 12:58:12 PM | CREATE_IN_PROGRESS   | AWS::IAM::Role              | Pipeline/line/Assets/3f399ce66b41b1c428a01d8b7eeaa7a3d41331c4b14d39325e6708e7d37836f0/PipelineActionRole elineAssets3f399ce66b41b1c428a01d8b7eeaa7a3d41331c4b14d39325e6708e7d37836f0CodePipelineAcRole9984FB3F) Resource creation Initiated
  3/33 | 12:58:12 PM | CREATE_IN_PROGRESS   | AWS::IAM::Role              | Pipeline/line/Assets/89524a47ac4311f937565f59b5bc547cc948ad206a9a3f668dfc9bf6c8a8a604/PipelineActionRole elineAssets89524a47ac4311f937565f59b5bc547cc948ad206a9a3f668dfc9bf6c8a8a604CodePipelineAcRole0A88A03D) 
  3/33 | 12:58:12 PM | CREATE_IN_PROGRESS   | AWS::IAM::Role              | Pipeline/line/Assets/5ed45fb3355317663862b23a1c32016178ae9b6d91c7ad301fcdaf93a3dc66f3/PipelineActionRole elineAssets5ed45fb3355317663862b23a1c32016178ae9b6d91c7ad301fcdaf93a3dc66f3CodePipelineAcRoleD7224BDE) 
  3/33 | 12:58:12 PM | CREATE_IN_PROGRESS   | AWS::IAM::Role              | Pipeline/line/Assets/e3c385af9f0538d61e567e307305627f27b709bbe58c8efce25debed54d3ab21/PipelineActionRole elineAssetse3c385af9f0538d61e567e307305627f27b709bbe58c8efce25debed54d3ab21CodePipelineAcRole125C1262) Resource creation Initiated
  3/33 | 12:58:12 PM | UPDATE_IN_PROGRESS   | AWS::CodeBuild::Project     | Pipeline/ts/FileAsset7/Default (PipelineAssetsFileAsset7A51C54D0) 
  3/33 | 12:58:12 PM | CREATE_IN_PROGRESS   | AWS::IAM::Role              | Pipeline/line/Assets/89524a47ac4311f937565f59b5bc547cc948ad206a9a3f668dfc9bf6c8a8a604/PipelineActionRole elineAssets89524a47ac4311f937565f59b5bc547cc948ad206a9a3f668dfc9bf6c8a8a604CodePipelineAcRole0A88A03D) Resource creation Initiated
  3/33 | 12:58:12 PM | CREATE_IN_PROGRESS   | AWS::IAM::Role              | Pipeline/line/Assets/b1ffd606562b3b3604283f5eb0bf49884fd75410d1aa4ce32340a01fb2027afe/PipelineActionRole elineAssetsb1ffd606562b3b3604283f5eb0bf49884fd75410d1aa4ce32340a01fb2027afeCodePipelineAcRoleB5BD1B02) 
  3/33 | 12:58:12 PM | CREATE_IN_PROGRESS   | AWS::IAM::Role              | Pipeline/line/Assets/5ed45fb3355317663862b23a1c32016178ae9b6d91c7ad301fcdaf93a3dc66f3/PipelineActionRole elineAssets5ed45fb3355317663862b23a1c32016178ae9b6d91c7ad301fcdaf93a3dc66f3CodePipelineAcRoleD7224BDE) Resource creation Initiated
  3/33 | 12:58:12 PM | UPDATE_IN_PROGRESS   | AWS::CodeBuild::Project     | Pipeline/ts/FileAsset5/Default (PipelineAssetsFileAsset5184A5C2F) 
  3/33 | 12:58:13 PM | UPDATE_IN_PROGRESS   | AWS::CodeBuild::Project     | Pipeline/ts/FileAsset3/Default (PipelineAssetsFileAsset3FE71B523) 
  3/33 | 12:58:13 PM | CREATE_IN_PROGRESS   | AWS::IAM::Role              | Pipeline/line/Assets/b1ffd606562b3b3604283f5eb0bf49884fd75410d1aa4ce32340a01fb2027afe/PipelineActionRole elineAssetsb1ffd606562b3b3604283f5eb0bf49884fd75410d1aa4ce32340a01fb2027afeCodePipelineAcRoleB5BD1B02) Resource creation Initiated
  3/33 | 12:58:13 PM | UPDATE_IN_PROGRESS   | AWS::CodeBuild::Project     | Pipeline/ts/FileAsset4/Default (PipelineAssetsFileAsset474303B7D) 
  3/33 | 12:58:13 PM | UPDATE_IN_PROGRESS   | AWS::CodeBuild::Project     | Pipeline/ts/FileAsset9/Default (PipelineAssetsFileAsset9F08741A2) 
  3/33 | 12:58:13 PM | UPDATE_COMPLETE      | AWS::CodeBuild::Project     | Pipeline/ts/FileAsset7/Default (PipelineAssetsFileAsset7A51C54D0) 
  3/33 | 12:58:14 PM | UPDATE_COMPLETE      | AWS::CodeBuild::Project     | Pipeline/ts/FileAsset5/Default (PipelineAssetsFileAsset5184A5C2F) 
  3/33 | 12:58:14 PM | UPDATE_COMPLETE      | AWS::CodeBuild::Project     | Pipeline/ts/FileAsset4/Default (PipelineAssetsFileAsset474303B7D) 
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
  4/33 | 12:58:14 PM | UPDATE_COMPLETE      | AWS::CodeBuild::Project     | Pipeline/ts/FileAsset9/Default (PipelineAssetsFileAsset9F08741A2) 
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
  5/33 | 12:58:23 PM | UPDATE_COMPLETE      | AWS::CodeBuild::Project     | Pipeline/ts/FileAsset3/Default (PipelineAssetsFileAsset3FE71B523) 
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
 10/33 | 12:58:41 PM | CREATE_COMPLETE      | AWS::IAM::Role              | Pipeline/line/Assets/3f399ce66b41b1c428a01d8b7eeaa7a3d41331c4b14d39325e6708e7d37836f0/PipelineActionRole elineAssets3f399ce66b41b1c428a01d8b7eeaa7a3d41331c4b14d39325e6708e7d37836f0CodePipelineAcRole9984FB3F) 
 10/33 | 12:58:41 PM | CREATE_COMPLETE      | AWS::IAM::Role              | Pipeline/line/Assets/e3c385af9f0538d61e567e307305627f27b709bbe58c8efce25debed54d3ab21/PipelineActionRole elineAssetse3c385af9f0538d61e567e307305627f27b709bbe58c8efce25debed54d3ab21CodePipelineAcRole125C1262) 
 10/33 | 12:58:41 PM | CREATE_COMPLETE      | AWS::IAM::Role              | Pipeline/line/Assets/5ed45fb3355317663862b23a1c32016178ae9b6d91c7ad301fcdaf93a3dc66f3/PipelineActionRole elineAssets5ed45fb3355317663862b23a1c32016178ae9b6d91c7ad301fcdaf93a3dc66f3CodePipelineAcRoleD7224BDE) 
 10/33 | 12:58:41 PM | CREATE_COMPLETE      | AWS::IAM::Role              | Pipeline/line/Assets/89524a47ac4311f937565f59b5bc547cc948ad206a9a3f668dfc9bf6c8a8a604/PipelineActionRole elineAssets89524a47ac4311f937565f59b5bc547cc948ad206a9a3f668dfc9bf6c8a8a604CodePipelineAcRole0A88A03D) 
 10/33 | 12:58:41 PM | CREATE_COMPLETE      | AWS::IAM::Role              | Pipeline/line/Assets/b1ffd606562b3b3604283f5eb0bf49884fd75410d1aa4ce32340a01fb2027afe/PipelineActionRole elineAssetsb1ffd606562b3b3604283f5eb0bf49884fd75410d1aa4ce32340a01fb2027afeCodePipelineAcRoleB5BD1B02) 
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
 10/33 | 12:58:45 PM | CREATE_IN_PROGRESS   | AWS::IAM::Policy            | Pipeline/line/Assets/3f399ce66b41b1c428a01d8b7eeaa7a3d41331c4b14d39325e6708e7d37836f0/PipelineActionRole/DefaultPolicy elineAssets3f399ce66b41b1c428a01d8b7eeaa7a3d41331c4b14d39325e6708e7d37836f0CodePipelineAcRoleDefaultPolicy761E1831) 
 10/33 | 12:58:45 PM | CREATE_IN_PROGRESS   | AWS::IAM::Policy            | Pipeline/line/Assets/e3c385af9f0538d61e567e307305627f27b709bbe58c8efce25debed54d3ab21/PipelineActionRole/DefaultPolicy elineAssetse3c385af9f0538d61e567e307305627f27b709bbe58c8efce25debed54d3ab21CodePipelineAcRoleDefaultPolicyD25AD0A7) 
 10/33 | 12:58:46 PM | CREATE_IN_PROGRESS   | AWS::IAM::Policy            | Pipeline/line/Assets/89524a47ac4311f937565f59b5bc547cc948ad206a9a3f668dfc9bf6c8a8a604/PipelineActionRole/DefaultPolicy elineAssets89524a47ac4311f937565f59b5bc547cc948ad206a9a3f668dfc9bf6c8a8a604CodePipelineAcRoleDefaultPolicyA024FD64) 
 10/33 | 12:58:46 PM | CREATE_IN_PROGRESS   | AWS::IAM::Policy            | Pipeline/line/Assets/b1ffd606562b3b3604283f5eb0bf49884fd75410d1aa4ce32340a01fb2027afe/PipelineActionRole/DefaultPolicy elineAssetsb1ffd606562b3b3604283f5eb0bf49884fd75410d1aa4ce32340a01fb2027afeCodePipelineAcRoleDefaultPolicy2B847A5A) 
 10/33 | 12:58:46 PM | CREATE_IN_PROGRESS   | AWS::IAM::Policy            | Pipeline/line/Assets/3f399ce66b41b1c428a01d8b7eeaa7a3d41331c4b14d39325e6708e7d37836f0/PipelineActionRole/DefaultPolicy elineAssets3f399ce66b41b1c428a01d8b7eeaa7a3d41331c4b14d39325e6708e7d37836f0CodePipelineAcRoleDefaultPolicy761E1831) Resource creation Initiated
 10/33 | 12:58:46 PM | CREATE_IN_PROGRESS   | AWS::IAM::Policy            | Pipeline/line/Assets/e3c385af9f0538d61e567e307305627f27b709bbe58c8efce25debed54d3ab21/PipelineActionRole/DefaultPolicy elineAssetse3c385af9f0538d61e567e307305627f27b709bbe58c8efce25debed54d3ab21CodePipelineAcRoleDefaultPolicyD25AD0A7) Resource creation Initiated
 10/33 | 12:58:46 PM | CREATE_IN_PROGRESS   | AWS::IAM::Policy            | Pipeline/line/Assets/89524a47ac4311f937565f59b5bc547cc948ad206a9a3f668dfc9bf6c8a8a604/PipelineActionRole/DefaultPolicy elineAssets89524a47ac4311f937565f59b5bc547cc948ad206a9a3f668dfc9bf6c8a8a604CodePipelineAcRoleDefaultPolicyA024FD64) Resource creation Initiated
 10/33 | 12:58:47 PM | CREATE_IN_PROGRESS   | AWS::IAM::Policy            | Pipeline/line/Assets/b1ffd606562b3b3604283f5eb0bf49884fd75410d1aa4ce32340a01fb2027afe/PipelineActionRole/DefaultPolicy elineAssetsb1ffd606562b3b3604283f5eb0bf49884fd75410d1aa4ce32340a01fb2027afeCodePipelineAcRoleDefaultPolicy2B847A5A) Resource creation Initiated
 10/33 | 12:58:47 PM | UPDATE_IN_PROGRESS   | AWS::IAM::Policy            | Pipeline/line/Role/DefaultPolicy (PipelineRoleDefaultPolicy7BDC1ABB) 
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
 10/33 | 12:58:54 PM | CREATE_IN_PROGRESS   | AWS::IAM::Policy            | Pipeline/line/Assets/5ed45fb3355317663862b23a1c32016178ae9b6d91c7ad301fcdaf93a3dc66f3/PipelineActionRole/DefaultPolicy elineAssets5ed45fb3355317663862b23a1c32016178ae9b6d91c7ad301fcdaf93a3dc66f3CodePipelineAcRoleDefaultPolicy2F744749) 
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
 10/33 | 12:58:55 PM | CREATE_IN_PROGRESS   | AWS::IAM::Policy            | Pipeline/line/Assets/5ed45fb3355317663862b23a1c32016178ae9b6d91c7ad301fcdaf93a3dc66f3/PipelineActionRole/DefaultPolicy elineAssets5ed45fb3355317663862b23a1c32016178ae9b6d91c7ad301fcdaf93a3dc66f3CodePipelineAcRoleDefaultPolicy2F744749) Resource creation Initiated
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
 14/33 | 12:59:01 PM | CREATE_COMPLETE      | AWS::IAM::Policy            | Pipeline/line/Assets/e3c385af9f0538d61e567e307305627f27b709bbe58c8efce25debed54d3ab21/PipelineActionRole/DefaultPolicy elineAssetse3c385af9f0538d61e567e307305627f27b709bbe58c8efce25debed54d3ab21CodePipelineAcRoleDefaultPolicyD25AD0A7) 
 14/33 | 12:59:01 PM | CREATE_COMPLETE      | AWS::IAM::Policy            | Pipeline/line/Assets/3f399ce66b41b1c428a01d8b7eeaa7a3d41331c4b14d39325e6708e7d37836f0/PipelineActionRole/DefaultPolicy elineAssets3f399ce66b41b1c428a01d8b7eeaa7a3d41331c4b14d39325e6708e7d37836f0CodePipelineAcRoleDefaultPolicy761E1831) 
 14/33 | 12:59:02 PM | CREATE_COMPLETE      | AWS::IAM::Policy            | Pipeline/line/Assets/89524a47ac4311f937565f59b5bc547cc948ad206a9a3f668dfc9bf6c8a8a604/PipelineActionRole/DefaultPolicy elineAssets89524a47ac4311f937565f59b5bc547cc948ad206a9a3f668dfc9bf6c8a8a604CodePipelineAcRoleDefaultPolicyA024FD64) 
 14/33 | 12:59:02 PM | CREATE_COMPLETE      | AWS::IAM::Policy            | Pipeline/line/Assets/b1ffd606562b3b3604283f5eb0bf49884fd75410d1aa4ce32340a01fb2027afe/PipelineActionRole/DefaultPolicy elineAssetsb1ffd606562b3b3604283f5eb0bf49884fd75410d1aa4ce32340a01fb2027afeCodePipelineAcRoleDefaultPolicy2B847A5A) 
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
 15/33 | 12:59:10 PM | CREATE_COMPLETE      | AWS::IAM::Policy            | Pipeline/line/Assets/5ed45fb3355317663862b23a1c32016178ae9b6d91c7ad301fcdaf93a3dc66f3/PipelineActionRole/DefaultPolicy elineAssets5ed45fb3355317663862b23a1c32016178ae9b6d91c7ad301fcdaf93a3dc66f3CodePipelineAcRoleDefaultPolicy2F744749) 
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
 16/33 | 12:59:16 PM | UPDATE_COMPLETE      | AWS::IAM::Policy            | Pipeline/line/Role/DefaultPolicy (PipelineRoleDefaultPolicy7BDC1ABB) 
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
 17/33 | 12:59:21 PM | UPDATE_IN_PROGRESS   | AWS::CodePipeline::Pipeline | Pipeline/line (Pipeline9850B417) 
 17/33 | 12:59:23 PM | UPDATE_COMPLETE      | AWS::CodePipeline::Pipeline | Pipeline/line (Pipeline9850B417) 
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
 17/33 | 12:59:30 PM | UPDATE_COMPLETE_CLEA | AWS::CloudFormation::Stack  | lineStack 
Stack PipelineStack is still not stable (UPDATE_COMPLETE_CLEANUP_IN_PROGRESS)
 28/33 | 12:59:31 PM | DELETE_IN_PROGRESS   | AWS::IAM::Policy            | lineAssets5c35ab4bbb02012923f039e277f810740c0c1129c376e5473c4452672a457d4bCodePipelineActoleDefaultPolicyFA09C352 
 28/33 | 12:59:31 PM | DELETE_IN_PROGRESS   | AWS::IAM::Policy            | lineAssets57df2c208983103f0b0ad76b1f03a39c6c785c43c25ca42ba1a5513614ec8b0fCodePipelineActoleDefaultPolicy052FA284 
 28/33 | 12:59:31 PM | DELETE_IN_PROGRESS   | AWS::IAM::Policy            | lineAssets5c7d0a17549d0bf0e92e020035c8dd1a730757c6fca3ed654a9fa9e03add131aCodePipelineActoleDefaultPolicyA0973FFA 
 28/33 | 12:59:31 PM | DELETE_IN_PROGRESS   | AWS::IAM::Policy            | lineAssetsef20590f2c6f581a4ba83b5051974e11aa1da9c890c1177605cd23278aa8956cCodePipelineActoleDefaultPolicy58C825F4 
 28/33 | 12:59:31 PM | DELETE_IN_PROGRESS   | AWS::IAM::Policy            | lineAssets46c5549131686266a755813298e43f4061b42da642acf14c76d3ce6b785475a3CodePipelineActoleDefaultPolicy5F924066 
 28/33 | 12:59:32 PM | DELETE_COMPLETE      | AWS::IAM::Policy            | lineAssets57df2c208983103f0b0ad76b1f03a39c6c785c43c25ca42ba1a5513614ec8b0fCodePipelineActoleDefaultPolicy052FA284 
 28/33 | 12:59:32 PM | DELETE_COMPLETE      | AWS::IAM::Policy            | lineAssets5c7d0a17549d0bf0e92e020035c8dd1a730757c6fca3ed654a9fa9e03add131aCodePipelineActoleDefaultPolicyA0973FFA 
 28/33 | 12:59:32 PM | DELETE_COMPLETE      | AWS::IAM::Policy            | lineAssetsef20590f2c6f581a4ba83b5051974e11aa1da9c890c1177605cd23278aa8956cCodePipelineActoleDefaultPolicy58C825F4 
 28/33 | 12:59:32 PM | DELETE_COMPLETE      | AWS::IAM::Policy            | lineAssets5c35ab4bbb02012923f039e277f810740c0c1129c376e5473c4452672a457d4bCodePipelineActoleDefaultPolicyFA09C352 
 28/33 | 12:59:32 PM | DELETE_COMPLETE      | AWS::IAM::Policy            | lineAssets46c5549131686266a755813298e43f4061b42da642acf14c76d3ce6b785475a3CodePipelineActoleDefaultPolicy5F924066 
 28/33 | 12:59:32 PM | DELETE_IN_PROGRESS   | AWS::IAM::Role              | lineAssets57df2c208983103f0b0ad76b1f03a39c6c785c43c25ca42ba1a5513614ec8b0fCodePipelineActoleD66B82B1 
 28/33 | 12:59:32 PM | DELETE_IN_PROGRESS   | AWS::IAM::Role              | lineAssets5c35ab4bbb02012923f039e277f810740c0c1129c376e5473c4452672a457d4bCodePipelineActole5BE287C6 
 28/33 | 12:59:32 PM | DELETE_IN_PROGRESS   | AWS::IAM::Role              | lineAssetsef20590f2c6f581a4ba83b5051974e11aa1da9c890c1177605cd23278aa8956cCodePipelineActole9446E131 
 28/33 | 12:59:32 PM | DELETE_IN_PROGRESS   | AWS::IAM::Role              | lineAssets5c7d0a17549d0bf0e92e020035c8dd1a730757c6fca3ed654a9fa9e03add131aCodePipelineActole50CD21F4 
 28/33 | 12:59:33 PM | DELETE_IN_PROGRESS   | AWS::IAM::Role              | lineAssets46c5549131686266a755813298e43f4061b42da642acf14c76d3ce6b785475a3CodePipelineActole1B4B17CA 
 28/33 | 12:59:34 PM | DELETE_COMPLETE      | AWS::IAM::Role              | lineAssets5c35ab4bbb02012923f039e277f810740c0c1129c376e5473c4452672a457d4bCodePipelineActole5BE287C6 
 28/33 | 12:59:34 PM | DELETE_COMPLETE      | AWS::IAM::Role              | lineAssets57df2c208983103f0b0ad76b1f03a39c6c785c43c25ca42ba1a5513614ec8b0fCodePipelineActoleD66B82B1 
 28/33 | 12:59:34 PM | DELETE_COMPLETE      | AWS::IAM::Role              | lineAssets46c5549131686266a755813298e43f4061b42da642acf14c76d3ce6b785475a3CodePipelineActole1B4B17CA 
 28/33 | 12:59:34 PM | DELETE_COMPLETE      | AWS::IAM::Role              | lineAssetsef20590f2c6f581a4ba83b5051974e11aa1da9c890c1177605cd23278aa8956cCodePipelineActole9446E131 
 28/33 | 12:59:34 PM | DELETE_COMPLETE      | AWS::IAM::Role              | lineAssets5c7d0a17549d0bf0e92e020035c8dd1a730757c6fca3ed654a9fa9e03add131aCodePipelineActole50CD21F4 
 28/33 | 12:59:34 PM | UPDATE_COMPLETE      | AWS::CloudFormation::Stack  | lineStack 
Stack PipelineStack has completed updating

 ✅  PipelineStack

Stack ARN:
arn:aws:cloudformation:eu-central-1:1111111111:stack/PipelineStack/2d90-c68f-11ea-94ee-0a233b5d9d4c

[Container] 2020/07/15 12:59:37 Phase complete: BUILD State: SUCCEEDED
[Container] 2020/07/15 12:59:37 Phase context status code:  Message: 
[Container] 2020/07/15 12:59:37 Entering phase POST_BUILD
[Container] 2020/07/15 12:59:37 Phase complete: POST_BUILD State: SUCCEEDED
[Container] 2020/07/15 12:59:37 Phase context status code:  Message: 


<details>
<summary>CodeBuild logs</summary>

[Container] 2020/07/15 12:57:16 Running command cdk -a . deploy PipelineStack --require-approval=never --verboseCDK toolkit version: 1.51.0 (build 8c2d53c)Command line arguments: { _: [ 'deploy' ], a: '.', app: '.', 'require-approval': 'never', requireApproval: 'never', verbose: 1, v: 1, 'ignore-errors': false, ignoreErrors: false, json: false, j: false, ec2creds: undefined, i: undefined, 'version-reporting': undefined, versionReporting: undefined, 'path-metadata': true, pathMetadata: true, 'asset-metadata': true, assetMetadata: true, 'role-arn': undefined, r: undefined, roleArn: undefined, staging: true, 'no-color': false, noColor: false, fail: false, 'build-exclude': [], E: [], buildExclude: [], ci: false, execute: true, force: false, f: false, parameters: [ {} ], 'previous-parameters': true, previousParameters: true, '$0': '/usr/local/bin/cdk', STACKS: [ 'PipelineStack' ], stacks: [ 'PipelineStack' ] }merged settings: { versionReporting: true, pathMetadata: true, output: 'cdk.out', app: '.', context: {}, tags: [], assetMetadata: true, requireApproval: 'never', toolkitBucket: {}, staging: true }Toolkit stack: CDKToolkitSetting "CDK_DEFAULT_REGION" environment variable to eu-central-1Resolving default credentialsLooking up default account ID from STSDefault account ID: 1111111111Setting "CDK_DEFAULT_ACCOUNT" environment variable to 1111111111context: { 'aws:cdk:enable-path-metadata': true, 'aws:cdk:enable-asset-metadata': true }--app points to a cloud assembly, so we bypass synthPipelineStack: deploying...Assuming role 'arn:aws:iam::1111111111:role/hnb659fds-deploy-role-1111111111-eu-central-1'.Waiting for stack CDKToolkit to finish creating or updating...[0%] start: Publishing 65a6bd9bc7f71e9fd2a17c7f41bf3d87f620c2930b4ed93db84329371c27:1111111111-eu-central-1Retrieved account ID 1111111111 from disk cacheAssuming role 'arn:aws:iam::1111111111:role/hnb659fds-file-publishing-role-1111111111-eu-central-1'.[0%] check: Check s3://cdk-hnb659fds-assets-1111111111-eu-central-1/65a6bd9bc7f71e9fd2a17c7f41bf3d87f620c2930b4ed93db84329371c27[0%] upload: Upload s3://cdk-hnb659fds-assets-1111111111-eu-central-1/65a6bd9bc7f71e9fd2a17c7f41bf3d87f620c2930b4ed93db84329371c27[100%] success: Published 65a6bd9bc7f71e9fd2a17c7f41bf3d87f620c2930b4ed93db84329371c27:1111111111-eu-central-1PipelineStack: checking if we can skip deployPipelineStack: template has changedPipelineStack: deploying...Attempting to create ChangeSet CDK-5bd31a53-9245-454a-b8cf-60f8fc3afcb6 to update stack lineStackPipelineStack: creating CloudFormation changeset...
Initiated creation of changeset: aws:cloudformation:eu-central-1:1111111111:changeSet/5bd31a53-9245-454a-b8cf-60f8fc3afcb6/e2a256d5-c1d6-46b3-a202-9e9c9b106672; waiting for o finish creating...
Waiting for changeset CDK-5bd31a53-9245-454a-b8cf-60f8fc3afcb6 on stack PipelineStack to sh creating...
Changeset CDK-5bd31a53-9245-454a-b8cf-60f8fc3afcb6 on stack PipelineStack is still ting
Changeset CDK-5bd31a53-9245-454a-b8cf-60f8fc3afcb6 on stack PipelineStack is still ting
Changeset CDK-5bd31a53-9245-454a-b8cf-60f8fc3afcb6 on stack PipelineStack is still ting
Changeset CDK-5bd31a53-9245-454a-b8cf-60f8fc3afcb6 on stack PipelineStack is still ting
Initiating execution of changeset CDK-5bd31a53-9245-454a-b8cf-60f8fc3afcb6 on stack lineStack
Execution of changeset CDK-5bd31a53-9245-454a-b8cf-60f8fc3afcb6 on stack PipelineStack started; waiting for the update to complete...
Waiting for stack PipelineStack to finish creating or updating...
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS (User Initiated))
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
3/33 | 12:58:11 PM | CREATE_IN_PROGRESS | AWS::IAM::Role | Pipeline/line/Assets/3f399ce66b41b1c428a01d8b7eeaa7a3d41331c4b14d39325e6708e7d37836f0/PipelineActionRole elineAssets3f399ce66b41b1c428a01d8b7eeaa7a3d41331c4b14d39325e6708e7d37836f0CodePipelineAcRole9984FB3F)
3/33 | 12:58:12 PM | CREATE_IN_PROGRESS | AWS::IAM::Role | Pipeline/line/Assets/e3c385af9f0538d61e567e307305627f27b709bbe58c8efce25debed54d3ab21/PipelineActionRole elineAssetse3c385af9f0538d61e567e307305627f27b709bbe58c8efce25debed54d3ab21CodePipelineAcRole125C1262)
3/33 | 12:58:12 PM | CREATE_IN_PROGRESS | AWS::IAM::Role | Pipeline/line/Assets/3f399ce66b41b1c428a01d8b7eeaa7a3d41331c4b14d39325e6708e7d37836f0/PipelineActionRole elineAssets3f399ce66b41b1c428a01d8b7eeaa7a3d41331c4b14d39325e6708e7d37836f0CodePipelineAcRole9984FB3F) Resource creation Initiated
3/33 | 12:58:12 PM | CREATE_IN_PROGRESS | AWS::IAM::Role | Pipeline/line/Assets/89524a47ac4311f937565f59b5bc547cc948ad206a9a3f668dfc9bf6c8a8a604/PipelineActionRole elineAssets89524a47ac4311f937565f59b5bc547cc948ad206a9a3f668dfc9bf6c8a8a604CodePipelineAcRole0A88A03D)
3/33 | 12:58:12 PM | CREATE_IN_PROGRESS | AWS::IAM::Role | Pipeline/line/Assets/5ed45fb3355317663862b23a1c32016178ae9b6d91c7ad301fcdaf93a3dc66f3/PipelineActionRole elineAssets5ed45fb3355317663862b23a1c32016178ae9b6d91c7ad301fcdaf93a3dc66f3CodePipelineAcRoleD7224BDE)
3/33 | 12:58:12 PM | CREATE_IN_PROGRESS | AWS::IAM::Role | Pipeline/line/Assets/e3c385af9f0538d61e567e307305627f27b709bbe58c8efce25debed54d3ab21/PipelineActionRole elineAssetse3c385af9f0538d61e567e307305627f27b709bbe58c8efce25debed54d3ab21CodePipelineAcRole125C1262) Resource creation Initiated
3/33 | 12:58:12 PM | UPDATE_IN_PROGRESS | AWS::CodeBuild::Project | Pipeline/ts/FileAsset7/Default (PipelineAssetsFileAsset7A51C54D0)
3/33 | 12:58:12 PM | CREATE_IN_PROGRESS | AWS::IAM::Role | Pipeline/line/Assets/89524a47ac4311f937565f59b5bc547cc948ad206a9a3f668dfc9bf6c8a8a604/PipelineActionRole elineAssets89524a47ac4311f937565f59b5bc547cc948ad206a9a3f668dfc9bf6c8a8a604CodePipelineAcRole0A88A03D) Resource creation Initiated
3/33 | 12:58:12 PM | CREATE_IN_PROGRESS | AWS::IAM::Role | Pipeline/line/Assets/b1ffd606562b3b3604283f5eb0bf49884fd75410d1aa4ce32340a01fb2027afe/PipelineActionRole elineAssetsb1ffd606562b3b3604283f5eb0bf49884fd75410d1aa4ce32340a01fb2027afeCodePipelineAcRoleB5BD1B02)
3/33 | 12:58:12 PM | CREATE_IN_PROGRESS | AWS::IAM::Role | Pipeline/line/Assets/5ed45fb3355317663862b23a1c32016178ae9b6d91c7ad301fcdaf93a3dc66f3/PipelineActionRole elineAssets5ed45fb3355317663862b23a1c32016178ae9b6d91c7ad301fcdaf93a3dc66f3CodePipelineAcRoleD7224BDE) Resource creation Initiated
3/33 | 12:58:12 PM | UPDATE_IN_PROGRESS | AWS::CodeBuild::Project | Pipeline/ts/FileAsset5/Default (PipelineAssetsFileAsset5184A5C2F)
3/33 | 12:58:13 PM | UPDATE_IN_PROGRESS | AWS::CodeBuild::Project | Pipeline/ts/FileAsset3/Default (PipelineAssetsFileAsset3FE71B523)
3/33 | 12:58:13 PM | CREATE_IN_PROGRESS | AWS::IAM::Role | Pipeline/line/Assets/b1ffd606562b3b3604283f5eb0bf49884fd75410d1aa4ce32340a01fb2027afe/PipelineActionRole elineAssetsb1ffd606562b3b3604283f5eb0bf49884fd75410d1aa4ce32340a01fb2027afeCodePipelineAcRoleB5BD1B02) Resource creation Initiated
3/33 | 12:58:13 PM | UPDATE_IN_PROGRESS | AWS::CodeBuild::Project | Pipeline/ts/FileAsset4/Default (PipelineAssetsFileAsset474303B7D)
3/33 | 12:58:13 PM | UPDATE_IN_PROGRESS | AWS::CodeBuild::Project | Pipeline/ts/FileAsset9/Default (PipelineAssetsFileAsset9F08741A2)
3/33 | 12:58:13 PM | UPDATE_COMPLETE | AWS::CodeBuild::Project | Pipeline/ts/FileAsset7/Default (PipelineAssetsFileAsset7A51C54D0)
3/33 | 12:58:14 PM | UPDATE_COMPLETE | AWS::CodeBuild::Project | Pipeline/ts/FileAsset5/Default (PipelineAssetsFileAsset5184A5C2F)
3/33 | 12:58:14 PM | UPDATE_COMPLETE | AWS::CodeBuild::Project | Pipeline/ts/FileAsset4/Default (PipelineAssetsFileAsset474303B7D)
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
4/33 | 12:58:14 PM | UPDATE_COMPLETE | AWS::CodeBuild::Project | Pipeline/ts/FileAsset9/Default (PipelineAssetsFileAsset9F08741A2)
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
5/33 | 12:58:23 PM | UPDATE_COMPLETE | AWS::CodeBuild::Project | Pipeline/ts/FileAsset3/Default (PipelineAssetsFileAsset3FE71B523)
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
10/33 | 12:58:41 PM | CREATE_COMPLETE | AWS::IAM::Role | Pipeline/line/Assets/3f399ce66b41b1c428a01d8b7eeaa7a3d41331c4b14d39325e6708e7d37836f0/PipelineActionRole elineAssets3f399ce66b41b1c428a01d8b7eeaa7a3d41331c4b14d39325e6708e7d37836f0CodePipelineAcRole9984FB3F)
10/33 | 12:58:41 PM | CREATE_COMPLETE | AWS::IAM::Role | Pipeline/line/Assets/e3c385af9f0538d61e567e307305627f27b709bbe58c8efce25debed54d3ab21/PipelineActionRole elineAssetse3c385af9f0538d61e567e307305627f27b709bbe58c8efce25debed54d3ab21CodePipelineAcRole125C1262)
10/33 | 12:58:41 PM | CREATE_COMPLETE | AWS::IAM::Role | Pipeline/line/Assets/5ed45fb3355317663862b23a1c32016178ae9b6d91c7ad301fcdaf93a3dc66f3/PipelineActionRole elineAssets5ed45fb3355317663862b23a1c32016178ae9b6d91c7ad301fcdaf93a3dc66f3CodePipelineAcRoleD7224BDE)
10/33 | 12:58:41 PM | CREATE_COMPLETE | AWS::IAM::Role | Pipeline/line/Assets/89524a47ac4311f937565f59b5bc547cc948ad206a9a3f668dfc9bf6c8a8a604/PipelineActionRole elineAssets89524a47ac4311f937565f59b5bc547cc948ad206a9a3f668dfc9bf6c8a8a604CodePipelineAcRole0A88A03D)
10/33 | 12:58:41 PM | CREATE_COMPLETE | AWS::IAM::Role | Pipeline/line/Assets/b1ffd606562b3b3604283f5eb0bf49884fd75410d1aa4ce32340a01fb2027afe/PipelineActionRole elineAssetsb1ffd606562b3b3604283f5eb0bf49884fd75410d1aa4ce32340a01fb2027afeCodePipelineAcRoleB5BD1B02)
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
10/33 | 12:58:45 PM | CREATE_IN_PROGRESS | AWS::IAM::Policy | Pipeline/line/Assets/3f399ce66b41b1c428a01d8b7eeaa7a3d41331c4b14d39325e6708e7d37836f0/PipelineActionRole/DefaultPolicy elineAssets3f399ce66b41b1c428a01d8b7eeaa7a3d41331c4b14d39325e6708e7d37836f0CodePipelineAcRoleDefaultPolicy761E1831)
10/33 | 12:58:45 PM | CREATE_IN_PROGRESS | AWS::IAM::Policy | Pipeline/line/Assets/e3c385af9f0538d61e567e307305627f27b709bbe58c8efce25debed54d3ab21/PipelineActionRole/DefaultPolicy elineAssetse3c385af9f0538d61e567e307305627f27b709bbe58c8efce25debed54d3ab21CodePipelineAcRoleDefaultPolicyD25AD0A7)
10/33 | 12:58:46 PM | CREATE_IN_PROGRESS | AWS::IAM::Policy | Pipeline/line/Assets/89524a47ac4311f937565f59b5bc547cc948ad206a9a3f668dfc9bf6c8a8a604/PipelineActionRole/DefaultPolicy elineAssets89524a47ac4311f937565f59b5bc547cc948ad206a9a3f668dfc9bf6c8a8a604CodePipelineAcRoleDefaultPolicyA024FD64)
10/33 | 12:58:46 PM | CREATE_IN_PROGRESS | AWS::IAM::Policy | Pipeline/line/Assets/b1ffd606562b3b3604283f5eb0bf49884fd75410d1aa4ce32340a01fb2027afe/PipelineActionRole/DefaultPolicy elineAssetsb1ffd606562b3b3604283f5eb0bf49884fd75410d1aa4ce32340a01fb2027afeCodePipelineAcRoleDefaultPolicy2B847A5A)
10/33 | 12:58:46 PM | CREATE_IN_PROGRESS | AWS::IAM::Policy | Pipeline/line/Assets/3f399ce66b41b1c428a01d8b7eeaa7a3d41331c4b14d39325e6708e7d37836f0/PipelineActionRole/DefaultPolicy elineAssets3f399ce66b41b1c428a01d8b7eeaa7a3d41331c4b14d39325e6708e7d37836f0CodePipelineAcRoleDefaultPolicy761E1831) Resource creation Initiated
10/33 | 12:58:46 PM | CREATE_IN_PROGRESS | AWS::IAM::Policy | Pipeline/line/Assets/e3c385af9f0538d61e567e307305627f27b709bbe58c8efce25debed54d3ab21/PipelineActionRole/DefaultPolicy elineAssetse3c385af9f0538d61e567e307305627f27b709bbe58c8efce25debed54d3ab21CodePipelineAcRoleDefaultPolicyD25AD0A7) Resource creation Initiated
10/33 | 12:58:46 PM | CREATE_IN_PROGRESS | AWS::IAM::Policy | Pipeline/line/Assets/89524a47ac4311f937565f59b5bc547cc948ad206a9a3f668dfc9bf6c8a8a604/PipelineActionRole/DefaultPolicy elineAssets89524a47ac4311f937565f59b5bc547cc948ad206a9a3f668dfc9bf6c8a8a604CodePipelineAcRoleDefaultPolicyA024FD64) Resource creation Initiated
10/33 | 12:58:47 PM | CREATE_IN_PROGRESS | AWS::IAM::Policy | Pipeline/line/Assets/b1ffd606562b3b3604283f5eb0bf49884fd75410d1aa4ce32340a01fb2027afe/PipelineActionRole/DefaultPolicy elineAssetsb1ffd606562b3b3604283f5eb0bf49884fd75410d1aa4ce32340a01fb2027afeCodePipelineAcRoleDefaultPolicy2B847A5A) Resource creation Initiated
10/33 | 12:58:47 PM | UPDATE_IN_PROGRESS | AWS::IAM::Policy | Pipeline/line/Role/DefaultPolicy (PipelineRoleDefaultPolicy7BDC1ABB)
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
10/33 | 12:58:54 PM | CREATE_IN_PROGRESS | AWS::IAM::Policy | Pipeline/line/Assets/5ed45fb3355317663862b23a1c32016178ae9b6d91c7ad301fcdaf93a3dc66f3/PipelineActionRole/DefaultPolicy elineAssets5ed45fb3355317663862b23a1c32016178ae9b6d91c7ad301fcdaf93a3dc66f3CodePipelineAcRoleDefaultPolicy2F744749)
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
10/33 | 12:58:55 PM | CREATE_IN_PROGRESS | AWS::IAM::Policy | Pipeline/line/Assets/5ed45fb3355317663862b23a1c32016178ae9b6d91c7ad301fcdaf93a3dc66f3/PipelineActionRole/DefaultPolicy elineAssets5ed45fb3355317663862b23a1c32016178ae9b6d91c7ad301fcdaf93a3dc66f3CodePipelineAcRoleDefaultPolicy2F744749) Resource creation Initiated
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
14/33 | 12:59:01 PM | CREATE_COMPLETE | AWS::IAM::Policy | Pipeline/line/Assets/e3c385af9f0538d61e567e307305627f27b709bbe58c8efce25debed54d3ab21/PipelineActionRole/DefaultPolicy elineAssetse3c385af9f0538d61e567e307305627f27b709bbe58c8efce25debed54d3ab21CodePipelineAcRoleDefaultPolicyD25AD0A7)
14/33 | 12:59:01 PM | CREATE_COMPLETE | AWS::IAM::Policy | Pipeline/line/Assets/3f399ce66b41b1c428a01d8b7eeaa7a3d41331c4b14d39325e6708e7d37836f0/PipelineActionRole/DefaultPolicy elineAssets3f399ce66b41b1c428a01d8b7eeaa7a3d41331c4b14d39325e6708e7d37836f0CodePipelineAcRoleDefaultPolicy761E1831)
14/33 | 12:59:02 PM | CREATE_COMPLETE | AWS::IAM::Policy | Pipeline/line/Assets/89524a47ac4311f937565f59b5bc547cc948ad206a9a3f668dfc9bf6c8a8a604/PipelineActionRole/DefaultPolicy elineAssets89524a47ac4311f937565f59b5bc547cc948ad206a9a3f668dfc9bf6c8a8a604CodePipelineAcRoleDefaultPolicyA024FD64)
14/33 | 12:59:02 PM | CREATE_COMPLETE | AWS::IAM::Policy | Pipeline/line/Assets/b1ffd606562b3b3604283f5eb0bf49884fd75410d1aa4ce32340a01fb2027afe/PipelineActionRole/DefaultPolicy elineAssetsb1ffd606562b3b3604283f5eb0bf49884fd75410d1aa4ce32340a01fb2027afeCodePipelineAcRoleDefaultPolicy2B847A5A)
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
15/33 | 12:59:10 PM | CREATE_COMPLETE | AWS::IAM::Policy | Pipeline/line/Assets/5ed45fb3355317663862b23a1c32016178ae9b6d91c7ad301fcdaf93a3dc66f3/PipelineActionRole/DefaultPolicy elineAssets5ed45fb3355317663862b23a1c32016178ae9b6d91c7ad301fcdaf93a3dc66f3CodePipelineAcRoleDefaultPolicy2F744749)
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
16/33 | 12:59:16 PM | UPDATE_COMPLETE | AWS::IAM::Policy | Pipeline/line/Role/DefaultPolicy (PipelineRoleDefaultPolicy7BDC1ABB)
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
17/33 | 12:59:21 PM | UPDATE_IN_PROGRESS | AWS::CodePipeline::Pipeline | Pipeline/line (Pipeline9850B417)
17/33 | 12:59:23 PM | UPDATE_COMPLETE | AWS::CodePipeline::Pipeline | Pipeline/line (Pipeline9850B417)
Stack PipelineStack is still not stable (UPDATE_IN_PROGRESS)
17/33 | 12:59:30 PM | UPDATE_COMPLETE_CLEA | AWS::CloudFormation::Stack | lineStack
Stack PipelineStack is still not stable (UPDATE_COMPLETE_CLEANUP_IN_PROGRESS)
28/33 | 12:59:31 PM | DELETE_IN_PROGRESS | AWS::IAM::Policy | lineAssets5c35ab4bbb02012923f039e277f810740c0c1129c376e5473c4452672a457d4bCodePipelineActoleDefaultPolicyFA09C352
28/33 | 12:59:31 PM | DELETE_IN_PROGRESS | AWS::IAM::Policy | lineAssets57df2c208983103f0b0ad76b1f03a39c6c785c43c25ca42ba1a5513614ec8b0fCodePipelineActoleDefaultPolicy052FA284
28/33 | 12:59:31 PM | DELETE_IN_PROGRESS | AWS::IAM::Policy | lineAssets5c7d0a17549d0bf0e92e020035c8dd1a730757c6fca3ed654a9fa9e03add131aCodePipelineActoleDefaultPolicyA0973FFA
28/33 | 12:59:31 PM | DELETE_IN_PROGRESS | AWS::IAM::Policy | lineAssetsef20590f2c6f581a4ba83b5051974e11aa1da9c890c1177605cd23278aa8956cCodePipelineActoleDefaultPolicy58C825F4
28/33 | 12:59:31 PM | DELETE_IN_PROGRESS | AWS::IAM::Policy | lineAssets46c5549131686266a755813298e43f4061b42da642acf14c76d3ce6b785475a3CodePipelineActoleDefaultPolicy5F924066
28/33 | 12:59:32 PM | DELETE_COMPLETE | AWS::IAM::Policy | lineAssets57df2c208983103f0b0ad76b1f03a39c6c785c43c25ca42ba1a5513614ec8b0fCodePipelineActoleDefaultPolicy052FA284
28/33 | 12:59:32 PM | DELETE_COMPLETE | AWS::IAM::Policy | lineAssets5c7d0a17549d0bf0e92e020035c8dd1a730757c6fca3ed654a9fa9e03add131aCodePipelineActoleDefaultPolicyA0973FFA
28/33 | 12:59:32 PM | DELETE_COMPLETE | AWS::IAM::Policy | lineAssetsef20590f2c6f581a4ba83b5051974e11aa1da9c890c1177605cd23278aa8956cCodePipelineActoleDefaultPolicy58C825F4
28/33 | 12:59:32 PM | DELETE_COMPLETE | AWS::IAM::Policy | lineAssets5c35ab4bbb02012923f039e277f810740c0c1129c376e5473c4452672a457d4bCodePipelineActoleDefaultPolicyFA09C352
28/33 | 12:59:32 PM | DELETE_COMPLETE | AWS::IAM::Policy | lineAssets46c5549131686266a755813298e43f4061b42da642acf14c76d3ce6b785475a3CodePipelineActoleDefaultPolicy5F924066
28/33 | 12:59:32 PM | DELETE_IN_PROGRESS | AWS::IAM::Role | lineAssets57df2c208983103f0b0ad76b1f03a39c6c785c43c25ca42ba1a5513614ec8b0fCodePipelineActoleD66B82B1
28/33 | 12:59:32 PM | DELETE_IN_PROGRESS | AWS::IAM::Role | lineAssets5c35ab4bbb02012923f039e277f810740c0c1129c376e5473c4452672a457d4bCodePipelineActole5BE287C6
28/33 | 12:59:32 PM | DELETE_IN_PROGRESS | AWS::IAM::Role | lineAssetsef20590f2c6f581a4ba83b5051974e11aa1da9c890c1177605cd23278aa8956cCodePipelineActole9446E131
28/33 | 12:59:32 PM | DELETE_IN_PROGRESS | AWS::IAM::Role | lineAssets5c7d0a17549d0bf0e92e020035c8dd1a730757c6fca3ed654a9fa9e03add131aCodePipelineActole50CD21F4
28/33 | 12:59:33 PM | DELETE_IN_PROGRESS | AWS::IAM::Role | lineAssets46c5549131686266a755813298e43f4061b42da642acf14c76d3ce6b785475a3CodePipelineActole1B4B17CA
28/33 | 12:59:34 PM | DELETE_COMPLETE | AWS::IAM::Role | lineAssets5c35ab4bbb02012923f039e277f810740c0c1129c376e5473c4452672a457d4bCodePipelineActole5BE287C6
28/33 | 12:59:34 PM | DELETE_COMPLETE | AWS::IAM::Role | lineAssets57df2c208983103f0b0ad76b1f03a39c6c785c43c25ca42ba1a5513614ec8b0fCodePipelineActoleD66B82B1
28/33 | 12:59:34 PM | DELETE_COMPLETE | AWS::IAM::Role | lineAssets46c5549131686266a755813298e43f4061b42da642acf14c76d3ce6b785475a3CodePipelineActole1B4B17CA
28/33 | 12:59:34 PM | DELETE_COMPLETE | AWS::IAM::Role | lineAssetsef20590f2c6f581a4ba83b5051974e11aa1da9c890c1177605cd23278aa8956cCodePipelineActole9446E131
28/33 | 12:59:34 PM | DELETE_COMPLETE | AWS::IAM::Role | lineAssets5c7d0a17549d0bf0e92e020035c8dd1a730757c6fca3ed654a9fa9e03add131aCodePipelineActole50CD21F4
28/33 | 12:59:34 PM | UPDATE_COMPLETE | AWS::CloudFormation::Stack | lineStack
Stack PipelineStack has completed updating

✅ PipelineStack

Stack ARN:
arn:aws:cloudformation:eu-central-1:1111111111:stack/PipelineStack/2d90-c68f-11ea-94ee-0a233b5d9d4c

[Container] 2020/07/15 12:59:37 Phase complete: BUILD State: SUCCEEDED
[Container] 2020/07/15 12:59:37 Phase context status code: Message:
[Container] 2020/07/15 12:59:37 Entering phase POST_BUILD
[Container] 2020/07/15 12:59:37 Phase complete: POST_BUILD State: SUCCEEDED
[Container] 2020/07/15 12:59:37 Phase context status code: Message:

</details>

So the underlying issue (bug?) is the assets (or asset hashes) being updated in the 'Build' stage, even when the source code has not changed (same git commit).

I should maybe add that I'm using my own CodeBuild project to create the CDK synth (as outlined in the API reference) to build my lambdas and React SPA along with my CDK app.

const buildProject = new codebuild.PipelineProject(this, 'BuildProject', {
  buildSpec: codebuild.BuildSpec.fromSourceFilename('buildspec.yml'),
  environment: {
    buildImage: codebuild.LinuxBuildImage.STANDARD_4_0,
  },
});
const synthAction = new codepipeline_actions.CodeBuildAction({
  actionName: 'Build',
  project: buildProject,
  input: sourceArtifact,
  outputs: [cloudAssemblyArtifact],
});

const pipeline = new pipelines.CdkPipeline(this, 'Pipeline', {
  pipelineName: 'Pipeline',
  cloudAssemblyArtifact,
  sourceAction,
  synthAction,
});

My buildspec.yaml:

version: 0.2

phases:
  install:
    commands:
      # download binaries 
  pre_build:
    commands:
      - cd $CODEBUILD_SRC_DIR/infra/lambda # root of my Go lambdas
      - task deps
      - cd $CODEBUILD_SRC_DIR/web # root of React SPA
      - yarn install
      - cd $CODEBUILD_SRC_DIR/infra/ # root of my CDK app, which makes references to lambda and SPA build outputs produced in the "build" phase using lambda.AssetCode.fromAsset and s3deploy.Source.asset
      - yarn install
  build:
    commands:
      - cd $CODEBUILD_SRC_DIR/infra/lambda
      - task build package
      - cd $CODEBUILD_SRC_DIR/web
      - yarn build
      - yarn package:staging
      - yarn package:prod
      - cd $CODEBUILD_SRC_DIR/infra/
      - yarn build
      - yarn cdk synth

artifacts:
  base-directory: infra/cdk.out
  files:
    - '**/*'

I don't understand why my assets are being updated on every build. Clearly, Go and Webpack produce new (different) output files (different file creation dates, etc.) for the same input files on every build. But this shouldn't trigger CFN resource updates.

I hope to get help here because I'm currently stuck.

asterikx picture asterikx on 15 Jul 2020

I see similar behaviour in my pipeline:

Expected behaviour:
Pipeline only updates itself when there are infra updates.

Actual behaviour:
On each application code change, the pipeline self updates one time before deploying application artifacts.

Setup:
New pipelines
Lambdas which use code: code.fromAsset('./buildpath) and standardNpmSynth to trigger the build.

Possible explanation:
I didn't dive into the internals, but the only way I can explain this is that you're creating hashes for the artifact build dirs as well. So on first run of a new commit sha, the build output hash changes and you trigger a new pipeline update. On second run the hash is the same and you can proceed to deploymeny. This makes a deployment take a long time :)

nonken picture nonken on 15 Jul 2020

FWIW I had the same issue when running parcel to bundle my frontend code to be deployed with aws-s3-deployment. Turns out that parcel has a bug, where it includes the current directory into the hash. That creates new assets on every build, and causes the pipeline to update itself forever.

It seems that CDK shouldn't update the pipeline if an asset changes. The Asset stage should simply upload all assets in cdk.out, not create specific actions for each asset hash that require modifying the pipeline.

christophgysin picture christophgysin on 15 Jul 2020

Expected behaviour:
Pipeline only updates itself when there are infra updates.

It seems that CDK shouldn't update the pipeline if an asset changes. The Asset stage should simply upload all assets in cdk.out, not create specific actions for each asset hash that require modifying the pipeline.

I totally agree with @nonken and @christophgysin. As it turns out the hashes of my assets change with every build (Webpack generates JS chunks with short random IDs), so that's why the pipeline loops forever.
If CDK would just upload the "new" asset versions, everything would be fine.

asterikx picture asterikx on 15 Jul 2020
👍1

@asterikx I think I am facing a similar issue with webpack. However, my docker images also cause it to get rebuilt as well. So I think it is both.

seawatts picture seawatts on 17 Jul 2020

Having the same issue. In my build step I need to build my artifacts using Docker. The resulting asset hash constantly changes leading into the loop

bweigel picture bweigel on 18 Jul 2020

I'm having the same issue. I have a similar setup to what they describe in this blog: https://aws.amazon.com/blogs/developer/cdk-pipelines-continuous-delivery-for-aws-cdk-applications/

Any known workaround at this point?

ghost picture ghost on 20 Jul 2020

It seems that CDK shouldn't update the pipeline if an asset changes. The Asset stage should simply upload all assets in cdk.out, not create specific actions for each asset hash that require modifying the pipeline.

The current implementation creates a publishing action _for each asset_ in order to maximize parallelism. Assets are identified by a hash of their contents in order to enable heavy caching throughout the pipeline and automatically invalidate only the needed components.

The unfortunate side effect of this design is that any asset change results a mutation of the pipeline. We will need to explore approaches to avoid that (e.g. by creating publish actions based on the _number_ of assets and not on the actual asset hash). I am retitling this issue so we can follow up on this specific problem.

The other problem is if the asset hash is "unstable" - as in, a different hash is produced in every build. This may happen if the build output has e.g. timestamps in it and the hash is calculated on the bundle output. To circumvent that, the framework offers knobs to control how the hash is calculated. Namely [assetHashType] and [assetHash]. The former can be used with [asset bundling] to control if the hash is calculated on the _source_ or the _bundle_, and the latter can be used to specify a custom hash explicitly.

eladb picture eladb on 20 Jul 2020
👍2

@eladb thanks!

So it seems to me that for any compiled binaries or bundled files with random IDs or timestamps, it will be necessary to calculate the hash over the source files manually. E.g.

new s3deploy.BucketDeployment(this, 'DeploySite', {
  sources: [
    s3deploy.Source.asset(path.join(__dirname, '../web/dist'), {
      assetHash: cdk.FileSystem.fingerprint(path.join(__dirname, '../web'), { exclude: ['dist'] }),
    }),
  ],
  destinationBucket: siteBucket,
  distribution,
  distributionPaths: ['/index.html', '/runtime-config.js'],
});

Would it be possible to generate a token from the source path and use that as name for the publishing action, e.g. path.join(__dirname, '../web/dist') --> TOKEN 123 --> PublishAssets123. So hash changes would not require pipeline mutations. Though I'm not sure how the publish action would know which asset it should deploy in this case ...

asterikx picture asterikx on 20 Jul 2020

Would it be possible to generate a token from the source path and use that as name for the publishing action, e.g. path.join(__dirname, '../web/dist') --> TOKEN 123 --> PublishAssets123. So hash changes would not require pipeline mutations. Though I'm not sure how the publish action would know which asset it should deploy in this case ...

If we only use the path to calculate the hash, the asset won't be invalidated when it changes and will never be updated.

As I mentioned, I think we need to figure out a way to avoid pipeline updates when assets are changed and the way to do that would probably be to define publishing actions that are not coupled directly with the asset hash but rather with the number of assets in the app. If a new asset is added, the pipeline will be updated (another publish action is needed) but if assets are only updated, the pipeline won't need to change because we will already have a publishing action for each asset, and only the ones that changes will actually upload data (the rest will succeed with no-op).

eladb picture eladb on 21 Jul 2020
👍3
Was this page helpful?
0 / 5 - 0 ratings

Related issues

artyom-melnikov picture artyom-melnikov  Â·  3Comments
slipdexic picture slipdexic  Â·  3Comments
peterdeme picture peterdeme  Â·  3Comments
pepastach picture pepastach  Â·  3Comments
mirazmamun picture mirazmamun  Â·  3Comments