Terraform-provider-aws: Adding multiple subnets to Elasticsearch Resource with zone awareness

[ranvijayj]

Hi there,

Thank you for opening an issue. Please note that we try to keep the Terraform issue tracker reserved for bug reports and feature requests. For general usage questions, please see: https://www.terraform.io/community.html.

Terraform Version

Run terraform -v to show the version. If you are not running the latest version of Terraform, please upgrade because your issue may have already been fixed.

0.11

Affected Resource(s)

Please list the resources as a list, for example:

• opc_instance
• opc_storage_volume

AWS Elasticsearch domain

If this issue appears to affect multiple resources, it may be an issue with Terraform's core, so please mention this.

Terraform Configuration Files

module.vpc.public_subnets is a list of public subnets

enable_zone_awareness = true
.
.
.
 vpc_options {
    security_group_ids = ["${aws_security_group.elasticsearch.id}"]
    subnet_ids         = ["${module.vpc.public_subnets[0]}, ${module.vpc.public_subnets[1]}"]
  }

So, I try and pass subnets with list [0] and list [1]

Debug Output

Please provide a link to a GitHub Gist containing the complete debug output: https://www.terraform.io/docs/internals/debugging.html. Please do NOT paste the debug output in the issue; just paste a link to the Gist.

When I terraform apply below happens:

vpc_option
tags.Domain: "" => "dev2"
vpc_options.#: "" => "1"
vpc_options.0.availability_zones.#: "" => ""
vpc_options.0.security_group_ids.#: "" => "1"
vpc_options.0.security_group_ids.3025683553: "" => "sg-c48d2aba"
vpc_options.0.subnet_ids.#: "" => "1"
vpc_options.0.subnet_ids.4075328369: "" => "subnet-c9cf7fb0, subnet-b02dfefb"
vpc_options.0.vpc_id: "" => ""

1 error(s) occurred:

• aws_elasticsearch_domain.es: 1 error(s) occurred:
• aws_elasticsearch_domain.es: ValidationException: You must specify exactly two subnets.

Panic Output

If Terraform produced a panic, please provide a link to a GitHub Gist containing the output of the crash.log.

Expected Behavior

What should have happened?

How do I make ES subnets_ids accept 2 values from the list module.vpc.public_subnets? so as to enable zone awareness?

Actual Behavior

What actually happened?

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. terraform apply

Important Factoids

Are there anything atypical about your accounts that we should know? For example: Running in EC2 Classic? Custom version of OpenStack? Tight ACLs?

References

Are there any other GitHub issues (open or closed) or Pull Requests that should be linked here? For example:

• GH-1234

[catsby]

Hello –

Can you share how module.vpc.public_subnets is defined?
If it's a list, can you try subnet_ids = "${module.vpc.public_subnets}"?

[AndyMoore]

i literally just hit this problem and went looking for an answer this morning, and found this. my fix was:

-    subnet_ids = ["${var.db_subnet_ids[0]}, ${var.db_subnet_ids[1]}"]
+    subnet_ids = ["${var.db_subnet_ids[0]}", "${var.db_subnet_ids[1]}"]

which gives the output as:

  vpc_options.0.subnet_ids.3267634451:                        "" => "subnet-3223f079"
  vpc_options.0.subnet_ids.51930061:                          "" => "subnet-05c4747c"

[catsby]

I imagine subnet_ids = "${module.vpc.public_subnets}" would also work, maybe you could verify? Either way, I'm going to close this issue for now. Thanks!

[AndyMoore]

it needs a total of 2, so need to be explicit

[shivpathak]

Recently AWS announced to support 3 availability zones for Elasticsearch but I see terraform still lacking this feature, I think we need to add support of 3 explicit subnet support in terraform.

reference of aws announcement is: https://aws.amazon.com/about-aws/whats-new/2019/02/amazon-elasticsearch-service-now-supports-three-availability-zone-deployments/

[gposton]

@catsby can we re-open this given @shivpathak's comment above. AWS just released support for 3 AZ's, but it looks as if terraform still doesn't support it.

[gposton]

@catsby I tried https://github.com/terraform-providers/terraform-provider-aws/issues/4038#issuecomment-378871516 and https://github.com/terraform-providers/terraform-provider-aws/issues/4038#issuecomment-378871516. No luck.... I think this needs re-opened.

[kzw]

When I create the cluster using terraform I can confirm this error with provider aws 2.5.0 and terraform 0.11.13.

What is odd is that one can create a cluster using the web console and import the existing cluster into the same config with no problem.

[hdryx]

I'm having also the same problem, unable to use more than one subnet

vpc_options {
    subnet_ids = ["${data.aws_subnet.private_1.id}", "${data.aws_subnet.private_2.id}"]

    security_group_ids = ["${aws_security_group.es.id}"]
  }

Error: ValidationException: You must specify exactly one subnet.

I have terraform 0.12.1 and provider.aws v2.14.0

[hdryx]

Ok, we have to add set zone_awareness_enabled = true to be able to deploy on multiple subnets.

[shivpathak]

@catsby can we re-open this given @shivpathak's comment above. AWS just released support for 3 AZ's, but it looks as if terraform still doesn't support it.

Finally we got this (elasticsearch 3 az support in terraform) done. it's release in terraform-provider-aws version v2.20.0

reference linke - https://github.com/terraform-providers/terraform-provider-aws/pull/9398

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!