Terraform: Terraform 0.13.0 breaks `terraform init -get-plugins=false` flag

[ljfranklin]

Terraform Version

Terraform v0.13.0

Terraform Configuration Files

N/A

Debug Output

terraform init -get-plugins=false ...

          Initializing the backend...

          Successfully configured the backend "s3"! Terraform will automatically
          use this backend unless the backend configuration changes.

          Initializing provider plugins...
          - Finding latest version of -/custom...

          Error: Failed to query available provider packages

          Could not retrieve the list of available versions for provider -/custom:
          provider registry registry.terraform.io does not have a provider named
          registry.terraform.io/-/custom

Expected Behavior

Prior to Terraform 0.13.0, setting -get-plugins=false would skip trying to download plugins.

Actual Behavior

Terraform 0.13.0 now tries to download plugins even if setting -get-plugins=false.

Additional Context

The -get-plugins flag still exists, but its value is never read. This commit seems to have accidentally dropped support. We use this flag as part of our CI tooling: https://github.com/ljfranklin/terraform-resource.

[ljfranklin]

Tagging @apparentlymart as this commit introduces the breakage: https://github.com/hashicorp/terraform/commit/94e1ac2d07fba098af888f46355c5df31680fd92

[alisdair]

@ljfranklin Thanks for reporting this!

There's definitely something to fix here, although I'm not yet sure if that's removing the flag, or trying to make it work again. Can you say more about what you expect -get-plugins=false to do in this situation, and why that's useful to you?

[ljfranklin]

Thanks for taking a look @alisdair! Use case is a bit complicated, but essentially we have a shared backend config to manage statefile storage for multiple teams. We need to run terraform init followed by terraform workspace list to get a list of active workspaces without necessarily having access to the full set of terraform config files which are managed by each individual team. So our entire config would only contain:

terraform {
  backend "s3" {
    ...
  }
}

This works fine most of the time, but hits an issue if the default workspace contains custom plugins. In this case, running terraform init tries to download custom plugins referenced by the default workspace which we don't have since they are installed at a later time by the team that owns their config. Running terraform init -get-plugins=false gave us a graceful way to skip downloading providers if the default workspace referenced any custom providers. Original issue here. Keeping the -get-plugins=false behavior as it was prior to 0.13 would unblock our upgrade to Terraform 0.13.

[alisdair]

Thanks, that makes sense to me!

I think the reason we didn't re-implement the get-plugins option is probably because the installer mechanism changed so dramatically. We now support multiple possible sources, including filesystem mirrors (e.g. ~/.terraform.d/plugins), and it's not immediately obvious to me what the meaning of get-plugins=false ought to be for those sources.

One possible direction we might take to address this would be to extend the providercache.InstallMode type somehow. Perhaps adding InstallNone which effectively only checks that all required providers are installed, erroring if not.

[apparentlymart]

My sense of these various options on terraform init is that they are ways to work around the fact that terraform init is essentially overloading a set of distinct operations into a single command for convenience in the common case:

1. Validate and initialize the backend.
2. Install any remote modules the configuration requires.
3. Install any provider plugins the configuration requires.

Prior to Terraform 0.9 these were split across distinct commands, but in 0.9 we combined them all into terraform init because we'd seen that the separate operations were confusing and annoying in common Terraform usage. We included flags to disable various parts of it as a compromise to still allow doing these things separately in unusual cases, such as what @ljfranklin described here.

With that said then: my sense of -get-plugins=false is a request to skip step three above altogether, both making no changes to the installed plugins _and_ ignoring whether required plugins are present. A consequence of that would be that subsequently running terraform apply would produce an error saying that required plugins aren't available and that you need to run terraform init to install them. However, commands that don't interact with providers (which includes terraform workspace list) would still be able to work.

We've unfortunately not really made a conscious effort to maintain this opt-outs in the years since Terraform 0.9 changed the workflow, I guess because the unified command has made those unusual cases much less common than they were in 0.8 and earlier. We've also seen a number of challenges with this particular design over the years with these flags being opt-outs rather than opt-ins and thus introducing new behaviors can be breaking for those who don't want the new thing, so perhaps in the longer run we'll look for a different design to better support this "I want to initialize a backend and nothing else" sort of use-case, but in the meantime hopefully we can find a not-too-invasive way to reintroduce -get-plugins=false in a mode like I described above (skipping the plugin-related initialization steps altogether).

[artburkart]

Today, I ran into a situation where I get this error using Terraform v0.12.26:

$ terraform init
Initializing the backend...
Initializing provider plugins...
- Checking for available provider plugins...
Registry service unreachable.
This may indicate a network issue, or an issue with the requested Terraform Registry.
Error: registry service is unreachable, check https://status.hashicorp.com/ for status updates

And I get this error with Terraform v0.13.0

$ terraform init
Initializing the backend...
Initializing provider plugins...
- Finding latest version of hashicorp/null...
Error: Failed to query available provider packages
Could not retrieve the list of available versions for provider hashicorp/null:
could not connect to registry.terraform.io: Failed to request discovery
document: Get "https://registry.terraform.io/.well-known/terraform.json":
context deadline exceeded (Client.Timeout exceeded while awaiting headers)

I'm not sure what's currently wrong with the Terraform registry, but I was able to work around it in Terraform v0.12.26 by using the -get-plugins=false command, since I already had the providers I needed cached locally. On the other hand, I'm still stuck and unable to run terraform init with v0.13.0. Without reverse-engineering what a mirror looks like, I'm also unable to create a local filesystem mirror for v0.13.0:

$ terraform providers mirror ~/.terraform.d/plugins
- Mirroring hashicorp/null...

Error: Provider not available

Failed to download registry.terraform.io/hashicorp/null from its origin
registry: could not connect to registry.terraform.io: Failed to request
discovery document: Get
"https://registry.terraform.io/.well-known/terraform.json": net/http: request
canceled while waiting for connection (Client.Timeout exceeded while awaiting
headers).

It would've been nice if it were possible in v0.13.0 to circumvent the remote calls to registry.terraform.io until the registry problem is resolved.

[lorengordon]

For skipping plugin retrieval, my workaround has been to separately manage the plugin cache using terraform-bundle, extract the plugins from the bundle to TF_PLUGIN_CACHE_DIR, and use terraform init -plugin-dir="$TF_PLUGIN_CACHE_DIR"...