Terraform: Request: default -plugin-dir in terraformrc

[bclodius]

Most of the machines at my 9-5 do not have access to the internet without some sort of proxy with proxy authentication. I configured TF_PLUGIN_CACHE_DIR and downloaded the plugins once manually on our CI server but then realized that while this will help cache the plugins; terraform will still by default get some metadata unless I configure -plugin-dir per https://github.com/hashicorp/terraform/pull/16000.

If the maintainers are open I would like to take a stab at contributing this feature. This will prevent us from having to always specify -plugin-dir.

Terraform Version

0.11.1

Terraform Configuration Files

plugin_cache_dir   = "$HOME/.terraform.d/plugin-cache"
disable_checkpoint = true

Proposed Terraform Configuration Files

plugin_cache_dir   = "$HOME/.terraform.d/plugin-cache"
default_plugin_dir = "$HOME/.terraform.d/plugin-cache/linux_amd64"
disable_checkpoint = true

[apparentlymart]

Hi @bclodius!

This seems like a reasonable idea to me, though given that it will quite significantly change Terraform's behavior globally (the plugin auto-install behavior will be entirely disabled for all Terraform configurations) I think it's important to include an explicit message about this in the terraform init output so that users can understand why Terraform is not behaving in the default way. There is already a message to this effect when -plugin-dir is used on the command line, so I'd like to see a variant of this message that talks about the CLI configuration file instead.

Existing message when the CLI option is used:

terraform init -plugin-dir=/tmp/nope

Initializing provider plugins...

Missing required providers.

The following provider constraints are not met by the currently-installed
provider plugins:

* null (any version)

Terraform can automatically download and install plugins to meet the given
constraints, but this step was skipped due to the use of -get-plugins=false
and/or -plugin-dir on the command line.

If automatic installation is not possible or desirable in your environment,
you may manually install plugins by downloading a suitable distribution package
and placing the plugin's executable file in one of the directories given in
by -plugin-dir on the command line, or in the following directory if custom
plugin directories are not set:
    terraform.d/plugins/linux_amd64

Potential new message when default_plugin_dir is set:

terraform init -plugin-dir=/tmp/nope

Initializing provider plugins...

Missing required providers.

The following provider constraints are not met by the currently-installed
provider plugins:

* null (any version)

Terraform can automatically download and install plugins to meet the given
constraints, but this step was skipped due to the use of default_plugin_dir
in the CLI configuration file.

You can manually install plugins by downloading a suitable distribution package
and placing the plugin's executable file in the configured plugin directory:
    /usr/lib/terraform/plugins

Whereas the CLI-related error message is written with the possibility that the user might re-run _without_ -plugin-dir, I assumed here that if you've set default_plugin_dir in the CLI config then you've done it because automatic installation isn't possible, and so instead the message tells you which directory Terraform is looking at so the instructions are more directly actionable.

---

It sounds like an alternative solution for your root problem here would be for Terraform to support the usual HTTP_PROXY and HTTPS_PROXY environment variables, though perhaps that isn't sufficient in your case due to the need to also provide authentication to the proxy.

I believe we don't currently honor those environment variables because Terraform uses a "clean" HTTP client to avoid inheriting global settings configured by other http package callers linked in to Terraform, but it should be possible to do so with some suitable configuration of the separate HTTP clients used for this functionality. (Unfortunately I believe a separate client is used for fetching the metadata vs. fetching the actual files, so it will probably need some work in at least two places to get full support.)

[svenstaro]

As a Linux distribution maintainer, I'd very much like this as currently it's a big nuisance. We can't properly ship 3rd party plugins to users as system packages as it stands right now.