Aws-cdk: CodeCommit and CodePipeline in different stacks leads to circular dependency

[asterikx]

:question: General Issue

I want to have a CodeCommit repository in one stack triggering a CodePipeline in another stack, both stacks being deployed to the same region.

When passing the repo as a prop to the stack containing the pipeline, I get a circular dependency error:
'RepoStack' depends on 'PipelineStack' (RepoStack -> PipelineStack/Pipeline/Resource.Ref, RepoStack -> PipelineStack/Pipeline/EventsRole/Resource.Arn). Adding this dependency (PipelineStack -> RepoStack/Repository/Resource.Name) would create a cyclic reference.

My code looks as follows:

const repoStack = new RepoStack(app, 'RepoStack', {
  env: envDevOps,
});
new PipelineStack(app, 'PipelineStack', {
  env: envDevOps,
  repo: repoStack.repo,
});

export class RepoStack extends cdk.Stack {
  public readonly repo: codecommit.Repository;

  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    this.repo = new codecommit.Repository(this, 'Repository', {
      repositoryName: 'MyRepo',
    });
  }
}

export class PipelineStack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props: PipelineStackProps) {
    super(scope, id, props);

    const sourceAction = new codepipeline_actions.CodeCommitSourceAction({
      actionName: 'CodeCommit_Source',
      repository: props.repo,
      output: sourceOutput,
    });

    // ...
  }
}

The Question

How can I have a CodeCommit repository in one stack triggering a CodePipeline in another stack without introducing a circular dependency (both in the same account/region)?

When the RepoStack is deployed to another account, everything works fine. I suspect that, in this case, the cross-account support stacks do the magic.
Still, it comes to a surprise that support for cross-references across accounts is better than support for cross-references within the same account.

I think this problem can be generalized to many scenarios with circular dependencies.

Environment

• CDK CLI Version: 1.39.0 (build 5d727c1)
• OS: MacOS Catalina
• Language: all

[guywilsonjr]

I've run into similar problems. This is a dangerous hole. I think it cost me ~6 months of kicking until it works. Any variables used in the pipeline stack, must only reference the pipeline stack and are not allowed to be assigned to the main App.

I believe you want something closer to this:

new PipelineStack(app, 'PipelineStack', {
  env: envDevOps
});

export class PipelineStack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props: PipelineStackProps) {
    super(scope, id, props);
    const repoStack = new RepoStack(app, 'RepoStack', {
        env: envDevOps,
    });
    const sourceAction = new codepipeline_actions.CodeCommitSourceAction({
      actionName: 'CodeCommit_Source',
      repository: repoStack.repo,
      output: sourceOutput,
    });

    // ...
  }
}

[asterikx]

@guywilsonjr This would probably work without any issue.

But I'd rather try to avoid nested stacks. I also want to allow multiple pipelines (from different stacks) to access the same repository (e.g. separate pipelines for develop, staging, production - or - an infrastructure pipeline and a website pipeline).

My feature request is more about supporting cross-references _within the same region/account_ in a similar way than cross-references _between different regions/accounts_.

[guywilsonjr]

I see. I think I messed it up anyways. As far as I know all Pipeline related resources must be bound to the same cdk.Stack. Though you can define all of the resources in any file you want.
(source: https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-codecommit.Repository.html)
If you want to use it for multiple pipelines you should be using:

static fromRepositoryArn(scope, id, repositoryArn)
or
static fromRepositoryName(scope, id, repositoryName)

if the repository is already created. Where 'Scope' is your pipeline stack. You should be able to use that in all of your different pipelines
Will that work for you?

[brendonmatheson]

I've also hit this issue in the past few days with more or less the same configuration as in the original post while attempting to migrate a plain CloudFormation implementation that also had these resources split into separate stacks.

I dug into the CDK code a bit to try to understand the root cause and one of the causes appears to be the auto-created IAM resources such as the statement created in CodeCommitSourceAction.bound() as well as the auto-created CW Event - these end up being dependencies of the repo.

Agree with the comment by @asterikx that this is probably a generalizable issue. Anywhere that a construct makes changes to a resource that it does not directly own, there is potential for this to arise. There should be a clear rule enforced that constructs can only make changes to their own resources otherwise the dependencies will never be manageable. If you search for "cyclic" in the open issues you'll see a number of variants of this problem.

In this case it's convenient that the Pipeline constructs try to auto-create policies for me but I'd rather it fail fast and tell me I have to compose in another construct that creates the role so that I have more control and more transparency and so we don't get phantom dependencies.

[guywilsonjr]

I will add in complete agreement with @brendonmatheson I can honestly say I've spent at least 50 wasted hours(on the low end) on this problem. The true question is who is the most gracious and honorable typescript expert with time, energy, and enthusiasm to make it happen? 😉😉😜😜

[flashflexpro]

Yeah ,this is a bug, in root_stack:

                        new pipelineActions.CloudFormationCreateUpdateStackAction({
                            actionName: 'deploy_child_stack',
                            stackName: 'child_stack'
                        }),

will be recognized as a dependency root_stack -> child_stack, throw root_stackDeploychild_stackPipelineActionRole with policy:

            Action:
              - 'cloudformation:CreateStack'
....
              - 'cloudformation:ValidateTemplate'
            Effect: Allow
            Resource: !Join
              - ''
              - - 'arn:'
                - !Ref 'AWS::Partition'
                - ':cloudformation:'
                - !Ref 'AWS::Region'
                - ':'
                - !Ref 'AWS::AccountId'
                - ':stack/child_stack/*'`

then you can't reference from child_stack to root_stack anymore

This is definitely a bug, because root_stack does NOT really depend on child_stack here!