Aws-cdk: [@aws-cdk/pipelines] failed to import a vpc from a stack into application stage

[enricopesce]

A stack inside the CodePipeline application stage doesn't recognize an existing VPC

Reproduction Steps

export class ApplicationStage extends cdk.Stage {
    public readonly urlOutput: cdk.CfnOutput
    constructor(scope: cdk.Construct, id: string, props?: cdk.StageProps) {
        super(scope, id, props)
        const service = new ApplicationStack(this, id, config.VPC_NAME, props)
        this.urlOutput = service.urlOutput
    }
} 

export class ApplicationStack extends cdk.Stack {
    public readonly urlOutput: cdk.CfnOutput;

    constructor(scope: cdk.Construct, id: string, vpc_name: string, props?: cdk.StackProps) {
        super(scope, id, props)

        console.log("VPC NAME: ")
        console.log(JSON.stringify(vpc_name))
        console.log("PROPS: ")
        console.log(JSON.stringify(props))

        const vpc = ec2.Vpc.fromLookup(this, "vpc", { vpcName: vpc_name })

        console.log("VPC IMPORTED ")
        console.log(vpc)

        const sg = new ec2.SecurityGroup(this, 'sg', {
            vpc: vpc
        })

    }
}

What did you expect to happen?

I'm expecting the creation of the security group referenced to the existing VPC

What actually happened?

CloudFromation error:

The vpc ID 'vpc-12345' does not exist (Service: AmazonEC2; Status Code: 400; Error Code: InvalidVpcID.NotFound; Request ID: 1e003787-3b4f-4d13-8033-48fb609d4447; Proxy: null)

The console logs added for debugging inside the stack from the synth stage:

34 | VPC NAME:
35 | "VPC-RD"
36 | PROPS:
37 | {"env":{"account":"946412081729","region":"eu-west-1"}}
38 | VPC IMPORTED
39 | LookedUpVpc {
40 | node: ConstructNode {
41 | host: [Circular],
42 | _actualNode: Node {
43 | host: [Circular],
44 | _locked: false,
45 | _aspects: [],
46 | _children: [Object],
47 | _context: {},
48 | _metadata: [],
49 | _dependencies: Set {},
50 | invokedAspects: [],
51 | id: 'vpc',
52 | scope: [ApplicationStack]
53 | }
54 | },
55 | stack: ApplicationStack {
56 | node: ConstructNode { host: [Circular], _actualNode: [Node] },
57 | _missingContext: [ [Object] ],
58 | _stackDependencies: {},
59 | templateOptions: {},
60 | _logicalIds: LogicalIDs { renames: {}, reverse: {} },
61 | account: '946412081729',
62 | region: 'eu-west-1',
63 | environment: 'aws://946412081729/eu-west-1',
64 | terminationProtection: undefined,
65 | _stackName: 'stg-stg',
66 | tags: TagManager {
67 | tags: Map {},
68 | priorities: Map {},
69 | initialTagPriority: 50,
70 | resourceTypeName: 'aws:cdk:stack',
71 | tagFormatter: KeyValueFormatter {},
72 | tagPropertyName: 'tags'
73 | },
74 | artifactId: 'awsomepipelinespipelinestgC171B70D',
75 | templateFile: 'awsomepipelinespipelinestgC171B70D.template.json',
76 | synthesizer: DefaultStackSynthesizer {
77 | props: {},
78 | files: {},
79 | dockerImages: {},
80 | _stack: [Circular],
81 | bucketName: 'cdk-hnb659fds-assets-946412081729-eu-west-1',
82 | repositoryName: 'cdk-hnb659fds-container-assets-946412081729-eu-west-1',
83 | _deployRoleArn: 'arn:${AWS::Partition}:iam::946412081729:role/cdk-hnb659fds-deploy-role-946412081729-eu-west-1',
84 | _cloudFormationExecutionRoleArn: 'arn:${AWS::Partition}:iam::946412081729:role/cdk-hnb659fds-cfn-exec-role-946412081729-eu-west-1',
85 | fileAssetPublishingRoleArn: 'arn:${AWS::Partition}:iam::946412081729:role/cdk-hnb659fds-file-publishing-role-946412081729-eu-west-1',
86 | imageAssetPublishingRoleArn: 'arn:${AWS::Partition}:iam::946412081729:role/cdk-hnb659fds-image-publishing-role-946412081729-eu-west-1',
87 | _kmsKeyArnExportName: 'CdkBootstrap-hnb659fds-FileAssetKeyArn'
88 | },
89 | [Symbol(@aws-cdk/core.DependableTrait)]: { dependencyRoots: [Array] }
90 | },
91 | env: { account: '946412081729', region: 'eu-west-1' },
92 | _physicalName: undefined,
93 | _allowCrossEnvironment: false,
94 | physicalName: '${Token[TOKEN.187]}',
95 | natDependencies: [],
96 | incompleteSubnetDefinition: true,
97 | internetConnectivityEstablished: ConcreteDependable {
98 | _dependencyRoots: [],
99 | [Symbol(@aws-cdk/core.DependableTrait)]: { dependencyRoots: [Getter] }
100 | },
101 | vpcId: 'vpc-12345',
102 | cidr: '1.2.3.4/5',
103 | _vpnGatewayId: undefined,
104 | availabilityZones: [ 'dummy1a', 'dummy1b' ],
105 | publicSubnets: [
106 | ImportedSubnet {
107 | node: [ConstructNode],
108 | stack: [ApplicationStack],
109 | env: [Object],
110 | _physicalName: undefined,
111 | _allowCrossEnvironment: false,
112 | physicalName: '${Token[TOKEN.188]}',
113 | internetConnectivityEstablished: [ConcreteDependable],
114 | _availabilityZone: 'dummy1a',
115 | subnetId: 's-12345',
116 | routeTable: [Object],
117 | [Symbol(@aws-cdk/core.DependableTrait)]: [Object]
118 | },
119 | ImportedSubnet {
120 | node: [ConstructNode],
121 | stack: [ApplicationStack],
122 | env: [Object],
123 | _physicalName: undefined,
124 | _allowCrossEnvironment: false,
125 | physicalName: '${Token[TOKEN.189]}',
126 | internetConnectivityEstablished: [ConcreteDependable],
127 | _availabilityZone: 'dummy1b',
128 | subnetId: 's-67890',
129 | routeTable: [Object],
130 | [Symbol(@aws-cdk/core.DependableTrait)]: [Object]
131 | }
132 | ],
133 | privateSubnets: [
134 | ImportedSubnet {
135 | node: [ConstructNode],
136 | stack: [ApplicationStack],
137 | env: [Object],
138 | _physicalName: undefined,
139 | _allowCrossEnvironment: false,
140 | physicalName: '${Token[TOKEN.190]}',
141 | internetConnectivityEstablished: [ConcreteDependable],
142 | _availabilityZone: 'dummy1a',
143 | subnetId: 'p-12345',
144 | routeTable: [Object],
145 | [Symbol(@aws-cdk/core.DependableTrait)]: [Object]
146 | },
147 | ImportedSubnet {
148 | node: [ConstructNode],
149 | stack: [ApplicationStack],
150 | env: [Object],
151 | _physicalName: undefined,
152 | _allowCrossEnvironment: false,
153 | physicalName: '${Token[TOKEN.191]}',
154 | internetConnectivityEstablished: [ConcreteDependable],
155 | _availabilityZone: 'dummy1b',
156 | subnetId: 'p-67890',
157 | routeTable: [Object],
158 | [Symbol(@aws-cdk/core.DependableTrait)]: [Object]
159 | }
160 | ],
161 | isolatedSubnets: [],
162 | [Symbol(@aws-cdk/core.DependableTrait)]: { dependencyRoots: [ [Circular] ] }
163 | }

Environment

• CLI Version : 1.61.1 (build 347918f)
• Framework Version: 1.60.0
• OS : aws/codebuild/standard:4.0
• *Language (Version): TypeScript (3.9.7)

Other

The full code is present on this repo https://github.com/enricopesce/AWSome-pipeline/tree/pipelines

---

This is :bug: Bug Report

[jpSimkins]

Could this be due to the quotes on the VPC name? I am using the vpcId and it works as expected.

    const vpc = Vpc.fromLookup(this, "existingVPC", {
      vpcId: "vpc-1234567890"
    });

In your console, you JSON.stringify(vpc_name) to show "VPC-RD", was there a reason for this? What is the output of vpc_name? Perhaps try using the same stringify when using the lookup.

[enricopesce]

Usually, I use the name is it easy to remember :) the same code in a manual stack deployed works correctly, the console log prints the name of the VPC in the same way with the console.log(vpc_name) I have printed the value of the variable for a test.

[enricopesce]

I have tested the lookup with vpcid but in both cases, the synth phase produce a template with "vpc-12345"

{
  "Resources": {
    "sg29196201": {
      "Type": "AWS::EC2::SecurityGroup",
      "Properties": {
        "GroupDescription": "awsome-pipelines-pipeline/stg/stg/sg",
        "SecurityGroupEgress": [
          {
            "CidrIp": "0.0.0.0/0",
            "Description": "Allow all outbound traffic by default",
            "IpProtocol": "-1"
          }
        ],
        "VpcId": "vpc-12345"
      },
      "Metadata": {
        "aws:cdk:path": "awsome-pipelines-pipeline/stg/stg/sg/Resource"
      }
    }
  }
}

If the same class is deployed directly from CDK as standard stack WORKS 100% correctly if this class is deployed by aws-pipelines as a stage not!!

[jguice]

This looks like the known limitation of context queries not being support in pipelines (#8905 tracks the feature request).

[enricopesce]

Thank you @jguice is it a very big limitation!

[rix0rrr]

Don't print the result of a fromLookup(). On the first iteration it's not going to contain what you think it does.

[sholtomaud]

This issue shouldn't be closed just yet @rix0rrr - we need a formal workaround for the limitation of context queries before CDK Pipelines can be used for an enterprise production solution. Happy to test against an actual use case.

[koups496]

I'm also looking for a workaround @rix0rrr

Is anyone using a workaround like this?

vpc = ec2.Vpc.from_vpc_attributes(self, 'VPC',
            vpc_id='vpc-xxxxxxxxx',
            availability_zones=["us-west-2a","us-west-2b","us-west-2c","us-west-2d"],
            private_subnet_ids=["subnet-xxxxxxxxxx", "subnet-xxxxxxxxxx", "subnet-xxxxxxxxxx", "subnet-xxxxxxxxxx"],

            )

I'm assuming most of us that are trying to use Vpc.fromLookup already know the vpc-id/az/subnet/etc information.

[akuma12]

This is currently the only reason we're not using CDK Pipelines. Would love to see this issue resolved.