Aws-cdk: ec2: cross-app usage of VPC

[pahud]

:question: General Issue

The Question

I am trying to build a sample to pass vpcId across stacks. It's fine if I pass vpcId as the stack property, however, the reality is, the vpc stack may be built by the infra team with native cloudformation templates and export the vpcId in the Outputs and the App team may build application with CDK in that VPC. All the App team knows is the export name and have to build an app stack in that VPC.

In TypeScript I can simply get the vpcId as String by cdk.Fn.importValue('ExportedVpcId')

import cdk = require('@aws-cdk/core');
import ec2 = require('@aws-cdk/aws-ec2');
import ecs = require('@aws-cdk/aws-ecs');
import ecsPatterns = require('@aws-cdk/aws-ecs-patterns');
import { ContainerImage } from '@aws-cdk/aws-ecs';
import { countResources } from '@aws-cdk/assert';
import { Vpc } from '@aws-cdk/aws-ec2';

export interface MyStackProps extends cdk.StackProps {
  vpc?: ec2.Vpc
  vpcId?: string
}

export class InfraStack extends cdk.Stack {
  readonly vpc: ec2.Vpc

  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    this.vpc = new ec2.Vpc(this, 'SharedVPC', {
      natGateways: 1,
    })

    new cdk.CfnOutput(this, 'vpcId', {
      value: this.vpc.vpcId,
      exportName: 'ExportedVpcId'
    })

  }
}

/**
 * This Fargate stack will be created within the ec2.Vpc we pass in from another stack
 */
export class FargateStack extends cdk.Stack {
  readonly vpc: ec2.Vpc

  constructor(scope: cdk.Construct, id: string, props: MyStackProps) {
    super(scope, id, props);

    this.vpc != props.vpc

    const cluster = new ecs.Cluster(this, 'Cluster', {
      vpc: this.vpc
    })

    const svc = new ecsPatterns.ApplicationLoadBalancedFargateService(this, 'FargateService', {
      cluster,
      image: ContainerImage.fromRegistry('nginx')
    })

    new cdk.CfnOutput(this, 'ServiceURL', {
      value: `http://${svc.loadBalancer.loadBalancerDnsName}/`
    })
  }
}

/**
 * This Fargate stack will be created within the vpcId we pass in from another stack
 */
export class FargateStack2 extends cdk.Stack {
  readonly vpc: string

  constructor(scope: cdk.Construct, id: string, props: MyStackProps) {
    super(scope, id, props);

    this.vpc != props.vpcId

    const cluster = new ecs.Cluster(this, 'Cluster', {
      vpc: Vpc.fromLookup(this, 'Vpc', {
        vpcId: this.vpc
      })
    })

    const svc = new ecsPatterns.ApplicationLoadBalancedFargateService(this, 'FargateService', {
      cluster,
      image: ContainerImage.fromRegistry('nginx')
    })

    new cdk.CfnOutput(this, 'ServiceURL', {
      value: `http://${svc.loadBalancer.loadBalancerDnsName}/`
    })
  }
}

And

/**
 * create our infra VPC
 */
const infra = new InfraStack(app, 'InfraStack', { env });

/**
 * create our Fargate service in the VPC from InfraStack
 */
const svc = new FargateStack(app, 'FargateServiceStack', {
    env,
    vpc: infra.vpc
})

/**
 * we can get the vpcId from the exported value from InfraStack
 */
const svc2 = new FargateStack2(app, 'FargateServiceStack2', {
    env,
    vpcId: cdk.Fn.importValue('ExportedVpcId')
})

However, in Python I got this error:

jsii.errors.JavaScriptError: 
  Error: All arguments to Vpc.fromLookup() must be concrete (no Tokens)

from aws_cdk import core, aws_ec2, aws_ecs, aws_ecs_patterns


class CdkPyCrossStackInfraStack(core.Stack):

    def __init__(self, scope: core.Construct, id: str, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        self.vpc = aws_ec2.Vpc(self, 'Vpc', nat_gateways=1)
        core.CfnOutput(self, 'vpcId', value=self.vpc.vpc_id, export_name='ExportedVpcId')


class CdkPyCrossStackFargateStack(core.Stack):
    def __init__(self, scope: core.Construct, id: str, vpc: aws_ec2.Vpc, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        self.vpc = vpc

        cluster = aws_ecs.Cluster(self, 'Cluster', vpc=self.vpc)
        svc = aws_ecs_patterns.ApplicationLoadBalancedFargateService(
            self, 'FargateService',
            cluster=cluster,
            image=aws_ecs.ContainerImage.from_registry('nginx'))

        core.CfnOutput(self, 'ServiceURL', value='http://{}/'.format(svc.load_balancer.load_balancer_full_name))


class CdkPyCrossStackFargateStack2(core.Stack):
    def __init__(self, scope: core.Construct, id: str, vpcId: str, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        cluster = aws_ecs.Cluster(self, 'Cluster', vpc=aws_ec2.Vpc.from_lookup(self, 'Vpc', vpc_id=vpcId))

        svc = aws_ecs_patterns.ApplicationLoadBalancedFargateService(
            self, 'FargateService',
            cluster=cluster,
            image=aws_ecs.ContainerImage.from_registry('nginx'))

        core.CfnOutput(self, 'ServiceURL', value='http://{}/'.format(svc.load_balancer.load_balancer_full_name))

infra_stack = CdkPyCrossStackInfraStack(app, "cdk-py-xstack-infra", env=AWS_ENV)
f1 = CdkPyCrossStackFargateStack(app, "cdk-py-xstack-fargate-svc1", env=AWS_ENV, vpc=infra_stack.vpc)
f2 = CdkPyCrossStackFargateStack2(app, "cdk-py-xstack-fargate-svc2",
                                  env=AWS_ENV,
                                  vpcId=core.Fn.import_value('ExportedVpcId')
                                  )


app.synth()

It looks like cdk.Fn.importValue() in TypeScript returns String but core.Fn.import_value() in Python returns Token.

Not sure if this is a valid issue. Any guidance is highly appreciated.

Environment

• CDK CLI Version: 1.8.0
• Module Version: 1.8.0
• OS: OSX Mojave
• Language: Python

Other information

https://gitter.im/awslabs/aws-cdk?at=5d7f7b8636461106bb29e96e

[shakerky]

I think I'm having this issue, but in Typescript.

This code fails:

new elb.ApplicationLoadBalancer(this, "loadBalancer", {
  internetFacing: true,
  idleTimeout: cdk.Duration.seconds(30),
  vpc: ec2.Vpc.fromLookup(this, 'vpcId',  {
   vpcId: cdk.Fn.importValue("VPCID")
  }),
})

Returns:

"All arguments to Vpc.fromLookup() must be concrete (no Tokens)"

Where this code works:

new elb.ApplicationLoadBalancer(this, "loadBalancer", {
  // http2Enabled: false,
  internetFacing: true,
  idleTimeout: cdk.Duration.seconds(30),
  vpc: ec2.Vpc.fromLookup(this, 'vpcId',  {
   vpcId: 'vpc-99999999' 
 }),
})

This is CDK 1.12.0

cdk --version
1.12.0 (build 923055e)

[gaddam1987]

I also face same issue when I try to use Fn.importValue("vpcId") in java cdk api

[quappinger]

Same here on TypeScript with CDK 1.12.0
Are there any updates on this issue?

[jstylen]

I am also experiencing this issue. Is there another way we are meant to pass the vpc id between stacks? I want to avoid having to add it in manually.

[RomainMuller]

That's a problem in _all_ languages, as in it is not (currently) possible to pass deploy-time values across environments that aren't co-located (same account AND region).

[rix0rrr]

And this use case can never really work. The reason is that we need to know more about a VPC than its VPC ID: we also need to know all of the subnets, routing tablets, etc, because there are a lot of things people want to do to VPCs and many of the things require detailed knowledge about the VPC layout.

The only way to get this going right now is for you infrastructure team to deploy the VPC completely, then use fromLookup() to find it in every region you intend to deploy to. I recommend using tags to identify the VPC.

[jishi]

And this use case can never really work. The reason is that we need to know more about a VPC than its VPC ID: we also need to know all of the subnets, routing tablets, etc, because there are a lot of things people want to do to VPCs and many of the things require detailed knowledge about the VPC layout.

The only way to get this going right now is for you infrastructure team to deploy the VPC completely, then use fromLookup() to find it in every region you intend to deploy to. I recommend using tags to identify the VPC.

But how come it works if you hard-code the VPC-id, but doesn't work when importing it? It would still not know anything about the remaining configuration?

[maschinetheist]

Is there any traction on this? As it stands it appears that exporting values from one stack to another simply does not work. I'm facing the same issue in Python.

[ThomasTaylorJr]

@maschinetheist As a work around, I used the boto3 library to pull in the export.

cf = boto3.client("cloudformation")
vpc_id = next(export['Value'] for export in cf.list_exports()['Exports'] if export['Name'] == 'export_name_here')

Hopefully though, this will be fixed.

[maschinetheist]

@maschinetheist As a work around, I used the boto3 library to pull in the export.

cf = boto3.client("cloudformation")
vpc_id = next(export['Value'] for export in cf.list_exports()['Exports'] if export['Name'] == 'export_name_here')

Hopefully though, this will be fixed.

Thank you!

I was able to make some progress as well, but without using exports:

class VPCStack(core.Stack):
    def __init__(self, app: core.App, id: str, **kwargs):
        super().__init__(app, id, **kwargs)

        self.vpc = aws_ec2.Vpc(self, "VPC", nat_gateways=1) 


class ConsumingStack(core.Stack):
    def __init__(self, app: core.App, id: str, vpc: aws_ec2.Vpc, **kwargs) -> None:
        super().__init__(app, id, **kwargs)

        # Update VPC routes
        self.vpc = vpc
        private_subnets = self.vpc.private_subnets
        for subnet in private_subnets:
            # print(subnet.route_table.route_table_id)
            route_table_stack = ec2.CfnRoute(
                self, str("MainRouteTable" + random.randint(0, 254)),
                route_table_id=subnet.route_table.route_table_id,
                destination_cidr_block="10.0.0.0/16"
                transit_gateway_id="tgw-123456"
            )


vpcstack = VPCStack(app, "VPCStack")
consumingstack = ConsumingStack(app, "ConsumingStack", vpc=vpcstack.vpc) # Import VPC construct from vpcstack
cdk.synth()

Of course this will work within an app (across stacks) but not cross-app. For that I will try out boto3.

[sheridansmall]

Having the same issue reading DistributionID from CloudFront to pass to stacks using another region. So we are using aws_cloudformation.CustomResource which calls a lambda to read the values we need. The same principal should work for VPC-id but it is a bit like using a sledge hammer to crack a nut.

[jones-chris]

I encountered this issue when getting the VPC ID from a CfnMapping object that is defined in the same stack:

mapping = core.CfnMapping(
            self, 'Mapping',
            mapping={
                'dev': {
                    'vpc': 'vpc-123456'
                },
                'test': {
                     'vpc': 'vpc-789012'
                 },
                 'prod': {
                     'vpc': 'vpc-345678'
                 }
            }

vpc = aws_ec2.Vpc.from_lookup(
                    self, 'Vpc',
                    vpc_id=mapping.find_in_map(env, 'vpc')
)

And I get this error:

jsii.errors.JavaScriptError: 
  Error: All arguments to Vpc.fromLookup() must be concrete (no Tokens)

But, hard coding the vpc works fine:

vpc = aws_ec2.Vpc.from_lookup(
              self, 'Vpc',
              vpc_id='vpc-123456'
)

[joehillen]

@jones-chris That mapping will only show up when cdk generates the cloudformation. You should use an if-statement or switch-statement.

[michaelday008]

I'm facing the same issue in typescript in CDK 1.31.0.

FYI, in this case, the VPC to be looked up is in the same account and region as the stack being deployed, and I'm passing the correct credentials for this account and region to the synthesis.

Below fails.

const vpcExportName = 'VPC-devops';
const vpcID = cdk.Fn.importValue(vpcExportName);
const codebuildVPC = ec2.Vpc.fromLookup(this, 'VPC', {
        vpcId: vpcID
      });  

below succeeds

const vpcID = '1234556';
const codebuildVPC = ec2.Vpc.fromLookup(this, 'VPC', {
        vpcId: vpcID
      });  

What is the exact problem here? Is the issue that the cross stack reference is not looked up at synthesis time, but expected to be looked up only at deploy time?

Is there a way that something can be implemented so that these cross stack references could be looked up at synthsis time?

const vpcID = cdk.Fn.importValueImmediate(vpcExportName);
or
const vpcID = cdk.Fn.importValueForSynthesis(vpcExportName);

It's a big pain to go through all my CDK templates when a VPC ID is changed and copy and paste the text value of the VPCID into the template. I would rather just go cdk synth them and have them pull in that new value automatically by referring to the stack output.

[michaeltarleton]

Is there a way that something can be implemented so that these cross stack references could be looked up at synthsis time?

For now my solution is to hard-code the VPC Name and do the lookup by vpc name rather than VPC ID. It's a temp fix in my eyes but necessary for me to move on w/ my project.

const vpcName = 'Some Constant VPC name across environments'
const vpc = ec2.Vpc.fromLookup(this, 'VPC', {
    vpcName
})

[markymarkus]

I bumped to this issue when trying to automate VPC CIDR allocation with CustomResource. When passing the cidr value from CustomResouce to L2 - Vpc construct I'm getting 'property must be a concrete CIDR string, got a Token '. Passing the same to L1 CfnVPC works just fine. Btw, I'm using python.

[hoegertn]

As michaelday008 pointed out the underlying issue here is that there are several phases a CDK app goes through on its journey to deployed stacks.

• Building the object model of L? constructs down to the tree with only L1 leaves
• Synthesizing a CloudFormation template out of these L1s
• Deploying this JSON file using the CloudFormation engine

These steps happen strongly one after the other. So information gathered in a later step can never be used in an earlier phase.

Now let's look at the steps described in this ticket and in which phase they happen.

_Object Model phase_

• Computing CIDR ranges for subnets, VPC, etc
• Looking up VPCs by id or name

_CFN Deployment_

• Resolving Mapping entries from CFN mappings using !FindInMap
• Importing values from CFN exports using !ImportValue
• Getting attributes of custom resources using !GetAtt

So whenever a piece of information in generated during the later phase it will be a Token in CDK and can never be used in one the action of the first phase.

This is why:

• VPC Lookups can only work with concrete ids, names etc and not with imports or mappings
• CIDR range calculation cannot work with outputs of custom resources
• Any other lookup with CFN deploy time values

So how to solve this?

For lookups you could, as already mentioned, always import by name or any other tag instead of using an id. Alternatively, you can always do the lookups, imports, etc in a script before invoking CDK and provide the values as context.

I hope this helps with some of the confusion of why certain things work and do not work and why it is not easily possible to "fix" this or why this is not a bug but working as intended.

[hoegertn]

I am not sure I can follow. Is the VPC created using the CDK app or not?

If it is created in the same app you can use the instance cross stack. IF not you should be able to look it up as it already exists.

Can you elaborate?