Terraform-provider-azurerm: Feature Request: HDinsight Cluster Deployment

[hashibot]

_This issue was originally opened by @retheshnair as hashicorp/terraform#11599. It was migrated here as part of the provider split. The original body of the issue is below._

---

Hi there,
I was trying to automate the deployment the HDinsight Cluster Deployment . I could not find terraform configuration to natively support HDinsight Cluster Deployment .Currently We are getting it deploy via ARM template + terraform based automation, which is not clean. It will nice to have HDinsight Cluster Deployment native support on terraform

[anoff]

Any info on what is missing to get this happen?

[seankanderson]

Are there any plans to implement HDInsight support?

[tombuildsstuff]

Related PR: #814

[AdamPitt]

@tombuildsstuff

Hello,

Could we please have an update on hd-insight? I thought from the milestone it would have been 1.7.0, is it in the latest master?

or is https://github.com/terraform-providers/terraform-provider-azurerm/pull/1267 the latest?

[rhollins]

Would be great to know when it will be implemented.
Thanks

[tombuildsstuff]

@AdamPitt progress is being tracked in #1267 :)

[1arrow]

Any update on this?

[tombuildsstuff]

@1arrow unfortunately not - this is being tracked in the PR (but this is blocked on the API at this time)

[tombuildsstuff]

👋

To give an update here: as mentioned in #1267 the HDInsight API is very generic (and thus the validation logic is complex/mostly client-side) - as such I believe we may need to take a different approach for supporting HDInsight Clusters in Terraform than has been taken in #1267.

After working with the API's for a while (and reflecting on it) - I believe it could make sense to have a Terraform Resource per HDInsight Cluster type (e.g. azurerm_hdinsight_(spark|storm|...)_cluster - which would allow the cluster definitions to become more solid, which in turn allows us to provide better validation/allows us to determine which fields can be ForceNew'd more effectively. This is going to require some prototyping/thought to determine how to proceed (as such I've added the "thinking" flag) - and temporarily closed #1267 for the moment.

[tombuildsstuff]

👋

To give an update here - after spending some time prototyping this the approach above (having one resource per kind) seems to provide a much better user experience than trying to make a generic resource. We're working on this at the moment - and as such we expect the following Data Sources/Resources to be required to make this work:

• [x] Data Source: azurerm_hdinsight_cluster (which will only expose the minimal common fields)
• [x] Resource: azurerm_hdinsight_kafka_cluster
• [x] Resource: azurerm_hdinsight_kbase_cluster
• [x] Resource: azurerm_hdinsight_hadoop_cluster
• [x] Resource: azurerm_hdinsight_interactive_query_cluster
• [x] Resource: azurerm_hdinsight_ml_services_cluster
• [x] Resource: azurerm_hdinsight_rserver_cluster
• [x] Resource: azurerm_hdinsight_spark_cluster
• [x] Resource: azurerm_hdinsight_storm_cluster

and then HDInsight Application (which can be done as a follow up) involves:

• [ ] Resource: azurerm_hdinsight_application

We'll post more information here as we've got it - but we're working on this at the moment :)

Thanks!

[tombuildsstuff]

Update: I've just opened #3196 which includes support for provisioning HDInsight Clusters; once that's merged we'll look to support HDInsight Applications next :)

[hashibot[bot]]

This has been released in version 1.25.0 of the provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading. As an example:

provider "azurerm" {
    version = "~> 1.25.0"
}
# ... other configuration ...

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!