Terragrunt: Reusable module for existing AWS resource

[kenzoawa]

Hello guys,

First of all i would like to say that terragrunt is awsome to use. Kudos for the gruntwork guys.
I have a kinda weird problem with logical separation of modules.

I have a existing VPC in which i want to deploy resources with terragrunt. I have defined the the code / rewrite that manually created VPC and after that i have terrgrunt import aws vpc into the state file. All works like a charm and i am successful in deploying in the existing VPC.

Now, when i do terragrunt destroy i don't want to remove the existing manually created resource, only the ones that i have added but i don't like writing terragrunt state rm (the existing one) everytime because it's prone to errors.

What i was thinking is that i like to have a module say VPC with his own state file and all other modules that i will be deploying to call that module and when i want to 'destroy' other modules i don't want that VPC to be deleted also. Does it make sense ? Does terragrunt will destroy that module also ?

Say in every other module that's depending on exiting VPC i need to put in terraform.tfvars ?

terraform { source = "../modules/vpc/" }

and in main.tf

module "existingvpc" { source = "/modules/vpc" }

If you guys have any other suggestion i would be grateful to take it as recommendation.

[yorinasub17]

Hi @kenzoawa,

If I understand your situation correctly, you have a different process that manages some resources (e.g in your example a manual process to manage VPCs), but need to refer to the attributes of the VPC in your terraform code. So to allow this, you are currently importing the VPC as a resource using terraform import. The problem with this is that now terraform is managing the VPC so it will attempt to change the attributes or destroy it, which you don't want because you have other processes to manage that resource. Is that correct?

If so, a better approach might be to use data sources, so that the terraform code isn't managing the resource, but is able to pull in the information.

[kenzoawa]

Hi @yorinasub17 thank you for the fast response.

Hi @kenzoawa,

If I understand your situation correctly, you have a different process that manages some resources (e.g in your example a manual process to manage VPCs), but need to refer to the attributes of the VPC in your terraform code. So to allow this, you are currently importing the VPC as a resource using terraform import. The problem with this is that now terraform is managing the VPC so it will attempt to change the attributes or destroy it, which you don't want because you have other processes to manage that resource. Is that correct?

Yes @yorinasub17 you are 100% correct.

If so, a better approach might be to use data sources, so that the terraform code isn't managing the resource, but is able to pull in the information.

Can i with data sources call/assign resources from that VPC module i have in terragrunt defined in other modules i have configured ? Let's say i want to use subnet from that VPC module for a new ASG group with data sources ?

I am still confused on how can i call resources from other modules in terragrunt without loading them into the current working module.

Thank you again for having the time to answer my questions.

[yorinasub17]

Can i with data sources call/assign resources from that VPC module i have in terragrunt defined in other modules i have configured ? Let's say i want to use subnet from that VPC module for a new ASG group with data sources ?

First, let me explain how this works in terraform, without terragrunt. The way to handle this would be to expose the VPC ID as an input variable. Then, you can lookup the subnets using data sources. For example, take a look at the following terraform code (note: this isn't complete, but should convey the idea):

variable "vpc_id" {}

resource "aws_instance" "web" {
  ami           = "${data.aws_ami.ubuntu.id}"
  instance_type = "t2.micro"
  subnet_id = "${element(data.aws_subnet_ids.vpc_subnets.ids, 0)}"
}

data "aws_subnet_ids" "vpc_subnets" {
  vpc_id = "${var.vpc_id}"
}

This looks up all the subnets in a VPC given the VPC ID as input to the module, and then uses the first one it finds to launch an EC2 instance into it.

Or, if you don't want to expose the ID, you can also lookup VPCs by name:

data "aws_vpc" "dev_vpc" {
  filter {
    name = "tag:Name"
    values = ["dev"]
  }
}

data "aws_subnet_ids" "vpc_subnets" {
  vpc_id = "${data.dev_vpc.id}"
}

Now let's say you want to do this using terragrunt. Well, all this still applies! You would do exactly the same thing in the underlying module, and then specify the input vpc_id in the terraform.tfvars file where you reference the module.

I am still confused on how can i call resources from other modules in terragrunt without loading them into the current working module.

Generally your code will be simpler if you can get away with raw data source lookups like above. However, sometimes you will want to reference more complex outputs from another module, which may not be possible to do with a data source. Or maybe you are using a newer resource that still doesn't have a data source.

One of the ways to get this to work is to use the terraform_remote_state data source. This data source let's you lookup data in another terraform state file. For example, if you had a directory structure as below in your live folder:

.
└── dev
    ├── app
    │   └── terraform.tfvars
    └── vpc
        └── terraform.tfvars

and you wanted to reference the outputs of the vpc module in your app module, you could have a terraform_remote_state lookup like below in you app module:

variable "terraform_state_aws_region" {
  description = "The region where the terraform state lives."
}

variable "terraform_state_s3_bucket" {
  description = "The bucket where the terraform state lives."
}

variable "environment" {
  description = "The infrastructure environment (e.g dev, stage, prod)"
}

data "terraform_remote_state" "vpc" {
  backend = "s3"
  config {
    region = "${var.terraform_state_aws_region}"
    bucket = "${var.terraform_state_s3_bucket}"
    key    = "${var.environment}/vpc/terraform.tfstate"
  }
}

Now data.terraform_remote_state.vpc contains all the outputs from the vpc module call in the same environment tree.

[kenzoawa]

Hey @yorinasub17

Thanks for the great input and trying to explain how terragrunt/terraform is working in the cases above, it's crystal clear now. I will definitely use some of the ways you describe above to call me resources and reference them in the repo.
I was just not sure what's the proper way / best practice with existing or new modules.

Thanks a lot!
Best regards,