Terraform: Terraform import within "resource" to import programmatically in the same repo (not as a command line utility but as an attribute within HCL)

[tolgaingenc]

Current Terraform Version

0.12.8

Use-cases

Having terraform import as a command line utility not always enough. We need to be able to import and manipulate resources which are created automatically like cloud9 instances or sometimes EMR instances and similar stuff.

For cloud9 for example you create a cloud9 instance resource (aws_cloud9_environment_ec2) and it creates and ec2 instance. I want to attached an IAM role or do any other manipulation. I can use data to address the instance which is just created but can't import it to manupulate.

Attempted Solutions

there are bad work arounds like manually importing outside of the repo or writing some kind of wrapper which will call terraform import and terraform apply

Proposal

Adding a new attribute to resource called resource_to_import or import which will import the resource instead of creating it if it wasn't imported already. It should start updating after importing like the import cli command.

resource "aws_cloud9_environment_ec2" "main" {
instance_type = "${var.cloud9_instance_type}"
name = "my_cloud9"
....
}

data "aws_instance" "cloud9" {
filter {
name = "tag:Name"
values = ["my_cloud9-*"]
}
}

resource "aws_instance" "my_instance" {
ami = "${var.ami_id}"
...
...
resource_to_import = "${data.aws_instance.cloud9.instance_id}"
}

References

No references

[rismoney]

What I like about this conceptually is it provides a means to import that is within a conventional workflow of terraform plan then apply.

terraform import doesn't quite fit into a normal pipeline. and requires interactive use. Typically this is handled inside a container inside a CI workflow. Having to interactively handle this or wrapper is heavy handed and defeats the point of all the automation.

[manpreetsnehra]

Another example :
EKS cluster is created and it creates certain resources with it.
To implement this https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts-cni-walkthrough.html its requried that that the aws-node service account is modified. But terraform cannot modify the aws-node service account so its not implementable.

So an implementation which is a simple
import "kubernetes_service_account" "aws-node" {
metadata {
name = "aws-node"
namespace = "kube-system"
}
or if no import
resource "kubernetes_service_account" "aws-node"{
modify = true/false
}

Where modify assumes that the resource exists and does the "import" and modify

The documentation says
"After adding inline IAM Policies (e.g. aws_iam_role_policy resource) or attaching IAM Policies (e.g. aws_iam_policy resource and aws_iam_role_policy_attachment resource) with the desired permissions to the IAM Role, annotate the Kubernetes service account (e.g. kubernetes_service_account resource) and recreate any pods."

But how to annonate a resource no created by terraform?

[winmillwill]

It would be nice for providers to have a sub-block like identification_policy where you could specify a set of labels that form a unique id, cluing terraform into your organization's conventions. Being able to guarantee that you know the identity of any resource before it's created would also allow things like detecting a renamed resource, like when you introduce for_each to handle resources that were separate before.

You may still want something else to capture the expectation that a resource already exists and avoid accidental creation.

[tolgaingenc]

But how to annonate a resource no created by terraform?

You define the resource anyway to use cli terraform import. So you will define the resource in your code and you will say import this and this time terraform apply will import this particular resource for you instead of creating it if it doesn't exist. But it should update if it exists.

So basically apply works in "Create if it doesn't exist, update if it exists" mode. This time, _if I set this flag and name the cloud resource_, it will work in "import if it doesn't exist, update if it exists" mode.

[shawnferry]

I have a similar problem with azure resources. Some resources "fail" but also create a resource.
If I plan and apply again I get the this resource must be imported errors. Problematic resources for me seem to regularly include azurerm_application_gateway and azurerm_app_service_custom_hostname_binding

I would like to see something like the following which could be added to problematic resources.

lifecycle {
 import_existing = true
}

Also the possibility of a command line to import for add all existing