Terraform-provider-aws: How to set EMR steps, similar to AWS::EMR::Step?

[hashibot]

_This issue was originally opened by @jitesh-prajapati as hashicorp/terraform#16133. It was migrated here as a result of the provider split. The original body of the issue is below._

---

How to set EMR steps, similar to AWS::EMR::Step?

[luckytaxi]

I see no reference to adding steps, does this not exist? Kind of useless for us if we can't add the steps during the creation of the cluster.

[ericzml]

Found the same issue, is there another to add the steps post cluster creation? But best case scenario is adding the steps as the cluster is being created.

[jitesh-prajapati]

how we can add steps during cluster creation even? I do not found anything in documentation.

[bacoboy]

dupe of #1593

[bflad]

I'll be working on a PR to implement this functionality. Related issues: #1593 and #1866

[bflad]

Support for configuring steps during cluster creation has been merged into master via #3673 and will release with v1.12.0 of the AWS provider, likely in about a week.

[bflad]

This has been released in version 1.12.0 of the AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

[kiran002]

Thanks for the feature guys, Additionally an example would be really appreciated!

Edit: added an example
step {
name="Hive program"
action_on_failure = "CONTINUE"
hadoop_jar_step {
jar="command-runner.jar"
args = ["hive-script","--run-hive-script",
"--hive-versions","2.3.2","--args","-f","s3://<bucket-name>/scripts/setup.sql"]
}
}

[richardxin]

does it support multiple steps?
i.e. I want to run another step when one completes.
For example, i want terminate the cluster once the step completes, since terraform currently don't support --auto-terminate according to https://github.com/terraform-providers/terraform-provider-aws/issues/1929

[hdryx]

does it support multiple steps?

Same question, does is possible to do multiple steps ?

[bflad]

Hi folks 👋

Yes, multiple steps are possible, e.g.

resource "aws_emr_cluster" "example" {
  # ... other configuration ...

  # Example from: https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-plan-debugging.html
  step {
    action_on_failure = "TERMINATE_CLUSTER"
    name              = "Setup Hadoop Debugging"

    hadoop_jar_step {
      jar  = "command-runner.jar"
      args = ["state-pusher-script"]
    }
  }

  # Example from: https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-spark-submit-step.html
  step {
    action_on_failure = "CONTINUE"
    name              = "Spark Step"

    hadoop_jar_step {
      jar  = "command-runner.jar"
      args = ["spark-example", "SparkPi", "10"]
    }
  }
}

Since this issue was closed long ago, going to lock it to encourage new bug/documentation/feature issues. 👍