Terraform: EMR Cluster is missing ServiceAccessSecurityGroup

[mdxp]

The sample code from aws_emr_cluster documentation will fail with

* aws_emr_cluster.emr_cluster: [WARN] Error waiting for EMR Cluster state to be "WAITING" 
or "RUNNING": unexpected state 'TERMINATED_WITH_ERRORS', wanted target 'WAITING, 
RUNNING'. last error: %!s(<nil>)

This is because when using the emr_managed_master_security_group and emr_managed_slave_security_group you also have to define and manage the ServiceAccessSecurityGroup (see cloudformation doc)

The AWS console will reveal the error and show:

Terminated with errors: You must also specify a ServiceAccessSecurityGroup 
if you use custom security groups when creating a cluster in a private subnet.

Terraform Version

0.7.7

Affected Resource(s)

• aws_emr_cluster
  

  Expected Behavior

There should be a ServiceAccessSecurityGroup attribute to be used and it should be mandatory to have it defined if one uses emr_managed_master_security_group and emr_managed_slave_security_group.

Steps to Reproduce

Any EMR cluster definition that uses emr_managed_master_security_group and emr_managed_slave_security_group (even the sample on the documentation) will fail as explained above. terraform apply will error out.

resource "aws_emr_cluster" "emr-test-cluster" {
  name          = "emr-test-arn"
  release_label = "emr-4.6.0"
  applications  = ["Spark"]

  ec2_attributes {
    subnet_id                         = "${aws_subnet.main.id}"
    emr_managed_master_security_group = "${aws_security_group.sg.id}"
    emr_managed_slave_security_group  = "${aws_security_group.sg.id}"
    instance_profile                  = "${aws_iam_instance_profile.emr_profile.arn}"
  }
...

[ddcprg]

Hello,

I opened a pull request https://github.com/hashicorp/terraform/pull/9600 yesterday with a patch for this issue - I didn't get any feedback in the user group about this issue so I opened it not being aware whether someone else was working on this. Are there any chances of getting this fixed in release 0.7.8?

Thanks,
Daniel

[mdxp]

Awesome. Thank you Daniel! Hopefully this can be merged in 0.7.8. This is a blocker for us on using terraform for emr.

[mdxp]

The fix has been merged in 0.7.10 and it works great. Closing.

[t-pascal]

I am seeing the same behaviour under 0.7.13. Is this a regression? I haven't tested 0.8.1 yet.

Error applying plan:

1 error(s) occurred:

* aws_emr_cluster.cluster: [WARN] Error waiting for EMR Cluster state to be "WAITING" or "RUNNING": unexpected state 'TERMINATING', wanted target 'WAITING, RUNNING'. last error: %!s(<nil>)

terraform -v
Terraform v0.7.13

[t-pascal]

Also, the docs need to be updated to include the correct code "Example bootable config" unless you want to keep it as an end-user testcase. Just kidding.

I'd recommend reopening this issue for the documentation (at the minimum) unless you want me to open a new separate issue.

[nickfyson]

I've hit the same problem as t-pascal when following the example!

[bam0382]

Same issue here...

[dkozlovskyi]

service_access_security_group ec2_attributes might help you solve this issue.

[eschwartz]

service_access_security_group ec2_attributes might help you solve this issue.

Can we document how to use this? I'm guessing we need to configure a new security group, to pass in
here. What rules does the SG need?

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.