Terraform: Add a simpler way to include third party / custom provider plugins

[nicksantamaria]

Terraform Version

$ terraform version
Terraform v0.11.2

Terraform Configuration Files

provider "bar" {
  # Usual provider connection / authentication config
  ... 

  # Proposed provider plugin distribution URL.
  provider_url = "https://github.com/foo/terraform-provider-bar/releases/download/v0.1.0/terraform-provider-bar-v0.1.0_${OS}_${ARCH}"
}

Expected Behavior

During terraform init terraform should fetch the provider plugin binary from the provider_url and cache it in the usual place inside .terraform directory.

Actual Behavior

Currently to install a custom or unsupported third-party binary, you have a few options:

• Preinstall the binary on the local filesystem, and use the -plugin-dir flag during initialisation (documented here).
• Install the provider binary to $HOME/.terraform.d/plugins (or windows equivalent) (documented here).
• Commit the .terraform/plugins directory within the project (can't find any documentation mentioning this, so I assume this is not a supported / best practice approach).

Additional Context

The current methods for installing third party / custom plugins has several usability issues.

• Installing a third party plugin is cumbersome
• As the two "recommended" approaches use shared directories on the system, there is the potential for versioning conflicts if multiple projects use a provider, but require differing versions.
• There is no official way to distribute / bundle a provider plugin with the terraform configuration.

[apparentlymart]

Hi @nicksantamaria! Thanks for this suggestion.

We are planning to support third-party plugin installation in the long run via new functionality in Terraform Registry and for private registries implementing its protocol. We plan to take this route because then we can fully support version constraints, selecting from builds for multiple target platforms, and features like automatic signature verification, which would be harder to achieve (require more complex configuration) for direct references to URLs as you proposed here.

There has been some work internally at HashiCorp to spec out the required registry functionality for this. The team that maintains the registry is currently focused elsewhere but support for provider plugins should come in the not-too-distant future, at which point we will also move the HashiCorp-distributed providers to be released via the same means.

---

While indeed the current recommended approaches are tricky and non-ideal, I just wanted to clarify that the system is designed to permit shared directories while _avoiding_ conflicts between multiple configurations: the plugin discovery system is able to recognize version numbers from the plugins and select appropriate ones in the same way as the auto-installer would do on releases.hashicorp.com.

In the case of your example, you'd name the binary terraform-provider-bar_v0.1.0, and then say version = "0.1.0" (or any constraint that matches) to select that particular instance.

[nicksantamaria]

Hi @apparentlymart,

Thanks for the detailed response, I'm glad to hear there are plans to improve this functionality!

Regarding private registries - are there any open source implementations of the protocol available?

I'll close this issue as the proposed solution is suboptimal compared to what you've described.

Cheers,
Nick

[pkoch]

Is there an issue tracking installing plugins from the registry (as opposed to just modules)?

[mildwonkey]

@pkoch: Yes! #15252. Thanks for the bump; I've assigned that issue to myself so I remember to post updates there!

[holms]

@mildwonkey any updates on this?

[mildwonkey]

@holms - yes, though I'd rather discuss this in the open ticket #15252 :) I will post a brief update there.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.