Terraform: bug?: 0.12.x -> 0.13.x: provider version in state: "Error: Could not load plugin"...

[tony-kerz]

Terraform Version

Terraform v0.13.1
+ provider registry.terraform.io/hashicorp/vault v2.13.0

Terraform Configuration Files

terraform {
  required_version = ">= 0.13"
  backend "s3" {}
  required_providers {
    vault = {
      source  = "hashicorp/vault"
      version = "~> 2.13.0"
    }
  }
}

provider "vault" {
  address = var.address
  token   = var.token
}

Expected Behavior

terraform allows provider versions referenced in state to be modified by newer providers in newer versions of code

Actual Behavior

, [2020-09-02T18:23:47.253410 #1]  INFO -- Tf: sh: cmd=[terraform init  -backend-config "profile=abc-mgmt-tf" -backend-config "bucket=abc-mgmt-tf-state" -backend-config "key=abc-tf-vault-tenant-arme.tfstate" -backend-config "dynamodb_table=tf-state-lock" -backend-config "encrypt=true" -backend-config "skip_metadata_api_check=true"], is-echo=true
Initializing modules...
- tenant in modules/vault/tenant

Initializing the backend...

Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.

Initializing provider plugins...
- Finding hashicorp/vault versions matching "~> 2.10"...
- Installing hashicorp/vault v2.13.0...
- Installed hashicorp/vault v2.13.0 (signed by HashiCorp)

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

I, [2020-09-02T18:23:51.224913 #1]  INFO -- Tf: sh: cmd=[terraform plan   -out tf.out], is-echo=true
rake aborted!
cmd=[terraform plan   -out tf.out] failed with status=[pid 75 exit 1]
/rake/module/util.rb:74:in `sh'
/rake/Rakefile:62:in `block (3 levels) in <top (required)>'
/rake/module/util.rb:27:in `inDir'
/rake/Rakefile:61:in `block (2 levels) in <top (required)>'
Tasks: TOP => tf:thru_plan => tf:plan
(See full trace by running task with --trace)

Error: Could not load plugin


Plugin reinitialization required. Please run "terraform init".

Plugins are external binaries that Terraform uses to access and manipulate
resources. The configuration provided requires plugins which can't be located,
don't satisfy the version constraints, or are otherwise incompatible.

Terraform automatically discovers provider requirements from your
configuration, including providers used in child modules. To see the
requirements and constraints, run "terraform providers".

Failed to instantiate provider "registry.terraform.io/-/vault" to obtain
schema: unknown provider "registry.terraform.io/-/vault"

Additional Context

I, [2020-09-02T18:29:12.614828 #1]  INFO -- Tf: sh: cmd=[terraform providers], is-echo=true

Providers required by configuration:
.
├── provider[registry.terraform.io/hashicorp/vault] ~> 2.10
└── module.tenant
    └── provider[registry.terraform.io/hashicorp/vault]

Providers required by state:

    provider[registry.terraform.io/-/vault]

[tony-kerz]

__update__: replace-provider seemed to do the trick, is this the appropriate action for this situation?

terraform state replace-provider -auto-approve registry.terraform.io/-/vault registry.terraform.io/hashicorp/vault

[apparentlymart]

Hi @tony-kerz! Sorry for this odd behavior and thanks for reporting it.

The terraform state replace-provider command is indeed the workaround I was going to suggest, and if that works for you then that should be fine to move on.

What seems odd here, and perhaps worthy of some more attention even though you've found a workaround, is that we expect terraform init to detect and install this legacy -/vault provider one time as part of the upgrade process, which would then normally allow the next terraform apply to implicitly do what that terraform state replace-provider command does and thus remove the last remant of -/vault so it'll no longer be needed on future terraform init.

In your case it seems like terraform init didn't try to install -/vault at all, which I think is unexpected. That then in turn left terraform plan in the awkward position of not having a provider that is clearly indicated in the state (which we can see in your terraform providers output).

I don't believe we've seen this particular situation -- where terraform init doesn't even attempt to install the legacy provider -- before. I wasn't able to quickly reproduce it with a similar configuration I constructed locally, but that doesn't really surprise me because I don't have your exact state snapshot and so my situation is expectedly different. If you're able to share it, it would be helpful to see the full log output that terraform init produces when you run it with the TF_LOG=trace environment variable set, because the trace logs include some more details about Terraform's process of deciding which providers it needs to install and what happened when it tried to install them.

[sethbacon]

See https://github.com/terraform-providers/terraform-provider-tfe/issues/209 and https://github.com/hashicorp/terraform/issues/26075 for additional examples.

[chroju]

I ran into the same problem with a provider registry.terraform.io/hashicorp/null . This provider already has a file path in registry.terraform.io/hashicorp , so the aforementioned workaround is not applicable for me :-/

Error: Could not load plugin


Plugin reinitialization required. Please run "terraform init".

Plugins are external binaries that Terraform uses to access and manipulate
resources. The configuration provided requires plugins which can't be located,
don't satisfy the version constraints, or are otherwise incompatible.

Terraform automatically discovers provider requirements from your
configuration, including providers used in child modules. To see the
requirements and constraints, run "terraform providers".

Failed to instantiate provider "registry.terraform.io/hashicorp/null" to
obtain schema: unknown provider "registry.terraform.io/hashicorp/null"

[fumantsu]

I have the same with aws and archive providers. I can't provide full trace logs but maybe these can help

Interesting also that after I used the replace-providers with successful planning, sequential planning and apply didn't found "matching provider" for the one with '-'

```
2020/09/24 17:51:47 [TRACE] providercache.fillMetaCache: scanning directory .terraform/plugins
2020/09/24 17:51:47 [TRACE] getproviders.SearchLocalDirectory: failed to resolve symlinks for .terraform/plugins: lstat .terraform/plugins: no such file or directory
2020/09/24 17:51:47 [TRACE] providercache.fillMetaCache: error while scanning directory .terraform/plugins: cannot search .terraform/plugins: lstat .terraform/plugins: no such file or directory
2020/09/24 17:51:47 [WARN] Failed to scan provider cache directory .terraform/plugins: cannot search .terraform/plugins: lstat .terraform/plugins: no such file or directory
2020/09/24 17:51:47 [DEBUG] Service discovery for registry.terraform.io at https://registry.terraform.io/.well-known/terraform.json
2020/09/24 17:51:47 [TRACE] HTTP client GET request to https://registry.terraform.io/.well-known/terraform.json
Initializing provider plugins...

• Finding hashicorp/archive versions matching "~> 1.3.0"...
  2020/09/24 17:51:47 [DEBUG] GET https://registry.terraform.io/v1/providers/hashicorp/archive/versions
  2020/09/24 17:51:47 [TRACE] HTTP client GET request to https://registry.terraform.io/v1/providers/hashicorp/archive/versions
• Finding hashicorp/aws versions matching "~> 2.66.0"...
  2020/09/24 17:51:47 [DEBUG] GET https://registry.terraform.io/v1/providers/hashicorp/aws/versions
  2020/09/24 17:51:47 [TRACE] HTTP client GET request to https://registry.terraform.io/v1/providers/hashicorp/aws/versions
  2020/09/24 17:51:47 [DEBUG] GET https://registry.terraform.io/v1/providers/hashicorp/archive/1.3.0/download/linux/amd64
  2020/09/24 17:51:47 [TRACE] HTTP client GET request to https://registry.terraform.io/v1/providers/hashicorp/archive/1.3.0/download/linux/amd64
  2020/09/24 17:51:47 [DEBUG] GET https://releases.hashicorp.com/terraform-provider-archive/1.3.0/terraform-provider-archive_1.3.0_SHA256SUMS
  2020/09/24 17:51:47 [TRACE] HTTP client GET request to https://releases.hashicorp.com/terraform-provider-archive/1.3.0/terraform-provider-archive_1.3.0_SHA256SUMS
  2020/09/24 17:51:47 [DEBUG] GET https://releases.hashicorp.com/terraform-provider-archive/1.3.0/terraform-provider-archive_1.3.0_SHA256SUMS.sig
  2020/09/24 17:51:47 [TRACE] HTTP client GET request to https://releases.hashicorp.com/terraform-provider-archive/1.3.0/terraform-provider-archive_1.3.0_SHA256SUMS.sig
  2020/09/24 17:51:47 [TRACE] providercache.Dir.InstallPackage: installing registry.terraform.io/hashicorp/archive v1.3.0 from https://releases.hashicorp.com/terraform-provider-archive/1.3.0/terraform-provider-archive_1.3.0_linux_amd64.zip
  2020/09/24 17:51:47 [TRACE] HTTP client GET request to https://releases.hashicorp.com/terraform-provider-archive/1.3.0/terraform-provider-archive_1.3.0_linux_amd64.zip
• Installing hashicorp/archive v1.3.0...
  2020/09/24 17:51:47 [DEBUG] Provider signed by 51852D87348FFC4C HashiCorp Security [e-mail]
  2020/09/24 17:51:48 [TRACE] providercache.fillMetaCache: scanning directory .terraform/plugins
  2020/09/24 17:51:48 [TRACE] getproviders.SearchLocalDirectory: .terraform/plugins is a symlink to .terraform/plugins
• Installed hashicorp/archive v1.3.0 (signed by HashiCorp)
  2020/09/24 17:51:48 [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/hashicorp/archive v1.3.0 for linux_amd64 at .terraform/plugins/registry.terraform.io/hashicorp/archive/1.3.0/linux_amd64
  2020/09/24 17:51:48 [TRACE] providercache.fillMetaCache: including .terraform/plugins/registry.terraform.io/hashicorp/archive/1.3.0/linux_amd64 as a candidate package for registry.terraform.io/hashicorp/archive 1.3.0
  2020/09/24 17:51:48 [DEBUG] GET https://registry.terraform.io/v1/providers/hashicorp/aws/2.66.0/download/linux/amd64
  2020/09/24 17:51:48 [TRACE] HTTP client GET request to https://registry.terraform.io/v1/providers/hashicorp/aws/2.66.0/download/linux/amd64
  2020/09/24 17:51:48 [DEBUG] GET https://releases.hashicorp.com/terraform-provider-aws/2.66.0/terraform-provider-aws_2.66.0_SHA256SUMS
  2020/09/24 17:51:48 [TRACE] HTTP client GET request to https://releases.hashicorp.com/terraform-provider-aws/2.66.0/terraform-provider-aws_2.66.0_SHA256SUMS
  2020/09/24 17:51:48 [DEBUG] GET https://releases.hashicorp.com/terraform-provider-aws/2.66.0/terraform-provider-aws_2.66.0_SHA256SUMS.sig
  2020/09/24 17:51:48 [TRACE] HTTP client GET request to https://releases.hashicorp.com/terraform-provider-aws/2.66.0/terraform-provider-aws_2.66.0_SHA256SUMS.sig
  2020/09/24 17:51:48 [TRACE] providercache.Dir.InstallPackage: installing registry.terraform.io/hashicorp/aws v2.66.0 from https://releases.hashicorp.com/terraform-provider-aws/2.66.0/terraform-provider-aws_2.66.0_linux_amd64.zip
  2020/09/24 17:51:48 [TRACE] HTTP client GET request to https://releases.hashicorp.com/terraform-provider-aws/2.66.0/terraform-provider-aws_2.66.0_linux_amd64.zip
• Installing hashicorp/aws v2.66.0...
  2020/09/24 17:51:48 [DEBUG] Provider signed by 51852D87348FFC4C HashiCorp Security [e-mail]
  2020/09/24 17:51:49 [TRACE] providercache.fillMetaCache: scanning directory .terraform/plugins
  2020/09/24 17:51:49 [TRACE] getproviders.SearchLocalDirectory: .terraform/plugins is a symlink to .terraform/plugins
• Installed hashicorp/aws v2.66.0 (signed by HashiCorp)
  2020/09/24 17:51:49 [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/hashicorp/archive v1.3.0 for linux_amd64 at .terraform/plugins/registry.terraform.io/hashicorp/archive/1.3.0/linux_amd64
  2020/09/24 17:51:49 [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/hashicorp/aws v2.66.0 for linux_amd64 at .terraform/plugins/registry.terraform.io/hashicorp/aws/2.66.0/linux_amd64
  2020/09/24 17:51:49 [TRACE] providercache.fillMetaCache: including .terraform/plugins/registry.terraform.io/hashicorp/archive/1.3.0/linux_amd64 as a candidate package for registry.terraform.io/hashicorp/archive 1.3.0
  2020/09/24 17:51:49 [TRACE] providercache.fillMetaCache: including .terraform/plugins/registry.terraform.io/hashicorp/aws/2.66.0/linux_amd64 as a candidate package for registry.terraform.io/hashicorp/aws 2.66.0
  2020/09/24 17:51:49 [TRACE] providercache.fillMetaCache: using cached result from previous scan of .terraform/plugins
  2020/09/24 17:51:49 [TRACE] providercache.fillMetaCache: using cached result from previous scan of .terraform/plugins
  Terraform has been successfully initialized!
  You may now begin working with Terraform. Try running "terraform plan" to see
  any changes that are required for your infrastructure. All Terraform commands
  should now work.
  If you ever set or change modules or backend configuration for Terraform,
  rerun this command to reinitialize your working directory. If you forget, other
  commands will detect it and remind you to do so if necessary.
  ....
  2020/09/24 17:51:50 [INFO] Terraform version: 0.13.3
  2020/09/24 17:51:50 [INFO] Go runtime version: go1.14.7
  2020/09/24 17:51:50 [INFO] CLI args: []string{"/usr/local/sbin/terraform013", "workspace", "list"}
  2020/09/24 17:51:50 [DEBUG] Attempting to open CLI config file: /home/gitlab-runner/.terraformrc
  2020/09/24 17:51:50 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
  2020/09/24 17:51:50 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
  2020/09/24 17:51:50 [DEBUG] ignoring non-existing provider search directory /home/gitlab-runner/.terraform.d/plugins
  2020/09/24 17:51:50 [DEBUG] ignoring non-existing provider search directory /home/gitlab-runner/.local/share/terraform/plugins
  2020/09/24 17:51:50 [DEBUG] ignoring non-existing provider search directory /usr/local/share/terraform/plugins
  2020/09/24 17:51:50 [DEBUG] ignoring non-existing provider search directory /usr/share/terraform/plugins
  2020/09/24 17:51:50 [INFO] CLI command args: []string{"workspace", "list"}
  2020/09/24 17:51:50 [TRACE] Meta.Backend: built configuration for "s3" backend with hash value 893988343
  2020/09/24 17:51:50 [TRACE] Preserving existing state lineage "e5da5c30-4cf3-2f22-db42-969816f1d73a"
  2020/09/24 17:51:50 [TRACE] Preserving existing state lineage "e5da5c30-4cf3-2f22-db42-969816f1d73a"
  2020/09/24 17:51:50 [TRACE] Meta.Backend: working directory was previously initialized for "s3" backend
  2020/09/24 17:51:50 [TRACE] Meta.Backend: using already-initialized, unchanged "s3" backend configuration
  ....
  2020/09/24 17:51:50 [TRACE] Meta.Backend: instantiated backend of type *s3.Backend
  2020/09/24 17:51:50 [TRACE] providercache.fillMetaCache: scanning directory .terraform/plugins
  2020/09/24 17:51:50 [TRACE] getproviders.SearchLocalDirectory: .terraform/plugins is a symlink to .terraform/plugins
  2020/09/24 17:51:50 [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/hashicorp/archive v1.3.0 for linux_amd64 at .terraform/plugins/registry.terraform.io/hashicorp/archive/1.3.0/linux_amd64
  2020/09/24 17:51:50 [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/hashicorp/aws v2.66.0 for linux_amd64 at .terraform/plugins/registry.terraform.io/hashicorp/aws/2.66.0/linux_amd64
  2020/09/24 17:51:50 [TRACE] providercache.fillMetaCache: including .terraform/plugins/registry.terraform.io/hashicorp/archive/1.3.0/linux_amd64 as a candidate package for registry.terraform.io/hashicorp/archive 1.3.0
  2020/09/24 17:51:50 [TRACE] providercache.fillMetaCache: including .terraform/plugins/registry.terraform.io/hashicorp/aws/2.66.0/linux_amd64 as a candidate package for registry.terraform.io/hashicorp/aws 2.66.0
  2020/09/24 17:51:51 [TRACE] providercache.fillMetaCache: using cached result from previous scan of .terraform/plugins
  2020/09/24 17:51:51 [DEBUG] checking for provisioner in "."
  2020/09/24 17:51:51 [DEBUG] checking for provisioner in "/usr/local/sbin"
  2020/09/24 17:51:51 [INFO] Failed to read plugin lock file .terraform/plugins/linux_amd64/lock.json: open .terraform/plugins/linux_amd64/lock.json: no such file or directory
  ...
  2020/09/24 17:51:53 [TRACE] Meta.Backend: instantiated backend of type *s3.Backend
  2020/09/24 17:51:53 [TRACE] providercache.fillMetaCache: scanning directory .terraform/plugins
  2020/09/24 17:51:53 [TRACE] getproviders.SearchLocalDirectory: .terraform/plugins is a symlink to .terraform/plugins
  2020/09/24 17:51:53 [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/hashicorp/archive v1.3.0 for linux_amd64 at .terraform/plugins/registry.terraform.io/hashicorp/archive/1.3.0/linux_amd64
  2020/09/24 17:51:53 [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/hashicorp/aws v2.66.0 for linux_amd64 at .terraform/plugins/registry.terraform.io/hashicorp/aws/2.66.0/linux_amd64
  2020/09/24 17:51:53 [TRACE] providercache.fillMetaCache: including .terraform/plugins/registry.terraform.io/hashicorp/archive/1.3.0/linux_amd64 as a candidate package for registry.terraform.io/hashicorp/archive 1.3.0
  2020/09/24 17:51:53 [TRACE] providercache.fillMetaCache: including .terraform/plugins/registry.terraform.io/hashicorp/aws/2.66.0/linux_amd64 as a candidate package for registry.terraform.io/hashicorp/aws 2.66.0
  2020/09/24 17:51:53 [TRACE] providercache.fillMetaCache: using cached result from previous scan of .terraform/plugins
  2020/09/24 17:51:54 [DEBUG] checking for provisioner in "."
  2020/09/24 17:51:54 [DEBUG] checking for provisioner in "/usr/local/sbin"
  2020/09/24 17:51:54 [INFO] Failed to read plugin lock file .terraform/plugins/linux_amd64/lock.json: open .terraform/plugins/linux_amd64/lock.json: no such file or directory
  2020/09/24 17:51:54 [TRACE] Meta.Backend: backend *s3.Backend does not support operations, so wrapping it in a local backend
  2020/09/24 17:51:54 [INFO] backend/local: starting Plan operation
  ...
  2020/09/24 17:51:54 [TRACE] backend/local: building context for current working directory
  2020/09/24 17:51:54 [TRACE] terraform.NewContext: starting
  2020/09/24 17:51:54 [TRACE] terraform.NewContext: loading provider schemas
  2020/09/24 17:51:54 [TRACE] LoadSchemas: retrieving schema for provider type "registry.terraform.io/hashicorp/archive"
  2020-09-24T17:51:54.378Z [INFO] plugin: configuring client automatic mTLS
  2020-09-24T17:51:54.416Z [DEBUG] plugin: starting plugin: path=.terraform/plugins/registry.terraform.io/hashicorp/archive/1.3.0/linux_amd64/terraform-provider-archive_v1.3.0_x4 args=[.terraform/plugins/registry.terraform.io/hashicorp/archive/1.3.0/linux_amd64/terraform-provider-archive_v1.3.0_x4]
  2020-09-24T17:51:54.421Z [DEBUG] plugin: plugin started: path=.terraform/plugins/registry.terraform.io/hashicorp/archive/1.3.0/linux_amd64/terraform-provider-archive_v1.3.0_x4 pid=28984
  2020-09-24T17:51:54.421Z [DEBUG] plugin: waiting for RPC address: path=.terraform/plugins/registry.terraform.io/hashicorp/archive/1.3.0/linux_amd64/terraform-provider-archive_v1.3.0_x4
  2020-09-24T17:51:54.433Z [INFO] plugin.terraform-provider-archive_v1.3.0_x4: configuring server automatic mTLS: timestamp=2020-09-24T17:51:54.432Z
  2020-09-24T17:51:54.473Z [DEBUG] plugin.terraform-provider-archive_v1.3.0_x4: plugin address: address=/tmp/plugin285926893 network=unix timestamp=2020-09-24T17:51:54.473Z
  2020-09-24T17:51:54.473Z [DEBUG] plugin: using plugin: version=5
  2020/09/24 17:51:54 [TRACE] GRPCProvider: GetSchema
  2020-09-24T17:51:54.552Z [TRACE] plugin.stdio: waiting for stdio data
  2020-09-24T17:51:54.553Z [WARN] plugin.stdio: received EOF, stopping recv loop: err="rpc error: code = Unimplemented desc = unknown service plugin.GRPCStdio"
  2020/09/24 17:51:54 [TRACE] No provider meta schema returned
  2020/09/24 17:51:54 [TRACE] GRPCProvider: Close
  2020-09-24T17:51:54.558Z [DEBUG] plugin: plugin process exited: path=.terraform/plugins/registry.terraform.io/hashicorp/archive/1.3.0/linux_amd64/terraform-provider-archive_v1.3.0_x4 pid=28984
  2020-09-24T17:51:54.558Z [DEBUG] plugin: plugin exited
  2020/09/24 17:51:54 [TRACE] LoadSchemas: retrieving schema for provider type "registry.terraform.io/hashicorp/aws"
  2020-09-24T17:51:54.558Z [INFO] plugin: configuring client automatic mTLS
  2020-09-24T17:51:54.598Z [DEBUG] plugin: starting plugin: path=.terraform/plugins/registry.terraform.io/hashicorp/aws/2.66.0/linux_amd64/terraform-provider-aws_v2.66.0_x4 args=[.terraform/plugins/registry.terraform.io/hashicorp/aws/2.66.0/linux_amd64/terraform-provider-aws_v2.66.0_x4]
  2020-09-24T17:51:54.600Z [DEBUG] plugin: plugin started: path=.terraform/plugins/registry.terraform.io/hashicorp/aws/2.66.0/linux_amd64/terraform-provider-aws_v2.66.0_x4 pid=28993
  2020-09-24T17:51:54.600Z [DEBUG] plugin: waiting for RPC address: path=.terraform/plugins/registry.terraform.io/hashicorp/aws/2.66.0/linux_amd64/terraform-provider-aws_v2.66.0_x4
  2020-09-24T17:51:54.629Z [INFO] plugin.terraform-provider-aws_v2.66.0_x4: configuring server automatic mTLS: timestamp=2020-09-24T17:51:54.628Z
  2020-09-24T17:51:54.668Z [DEBUG] plugin: using plugin: version=5
  2020-09-24T17:51:54.668Z [DEBUG] plugin.terraform-provider-aws_v2.66.0_x4: plugin address: address=/tmp/plugin180918770 network=unix timestamp=2020-09-24T17:51:54.667Z
  2020/09/24 17:51:54 [TRACE] GRPCProvider: GetSchema
  2020-09-24T17:51:54.749Z [TRACE] plugin.stdio: waiting for stdio data
  2020-09-24T17:51:54.759Z [WARN] plugin.stdio: received EOF, stopping recv loop: err="rpc error: code = Unimplemented desc = unknown service plugin.GRPCStdio"
  2020/09/24 17:51:54 [TRACE] No provider meta schema returned
  2020/09/24 17:51:54 [TRACE] GRPCProvider: Close
  2020-09-24T17:51:54.859Z [DEBUG] plugin: plugin process exited: path=.terraform/plugins/registry.terraform.io/hashicorp/aws/2.66.0/linux_amd64/terraform-provider-aws_v2.66.0_x4 pid=28993
  2020-09-24T17:51:54.859Z [DEBUG] plugin: plugin exited
  Error: Could not load plugin
  Plugin reinitialization required. Please run "terraform init".
  Plugins are external binaries that Terraform uses to access and manipulate
  resources. The configuration provided requires plugins which can't be located,
  don't satisfy the version constraints, or are otherwise incompatible.
  Terraform automatically discovers provider requirements from your
  configuration, including providers used in child modules. To see the
  requirements and constraints, run "terraform providers".
  2 problems:
• Failed to instantiate provider "registry.terraform.io/-/archive" to obtain
  schema: unknown provider "registry.terraform.io/-/archive"
• Failed to instantiate provider "registry.terraform.io/-/aws" to obtain
  schema: unknown provider "registry.terraform.io/-/aws"

[tysimo]

I'm seeing the same thing with the azurerm provider.

Terraform v0.13.4
Initializing plugins and modules...

Error: Could not load plugin

Plugin reinitialization required. Please run "terraform init".

Plugins are external binaries that Terraform uses to access and manipulate 
resources. The configuration provided requires plugins which can't be located,
don't satisfy the version constraints, or are otherwise incompatible.

Terraform automatically discovers provider requirements from your
configuration, including providers used in child modules. To see the
requirements and constraints, run "terraform providers".

Failed to instantiate provider "registry.terraform.io/-/azurerm" to obtain
schema: unknown provider "registry.terraform.io/-/azurerm"

[MuhammadShaban]

update: replace-provider seemed to do the trick, is this the appropriate action for this situation?

terraform state replace-provider -auto-approve registry.terraform.io/-/vault registry.terraform.io/hashicorp/vault

Thanks @tony-kerz, this workaround working for me

[Eraac]

terraform state replace-provider probably fix this, but when you've more than 100 state isn't a valid solution ...

[Schparky]

Dropping this here in case it can help in further troubleshooting or workarounds. The replace-provider workaround didn't work for me since we are using Terraform Cloud. Sort of a stab in the air, I changed the source from "hashicorp/aws" to -/aws:

terraform {
  required_version = ">= 0.13"
  required_providers {
    aws = {
      source = "-/aws"
    }
}

This dropped a mysterious error in the plan output:

Error: Missing required argument

The argument "region" is required, but was not set.

So I reverted the change and then the next plan and apply worked without error.

[Schparky]

I should have expected this, but my "workaround" didn't work on another workspace with the same defect.

[Schparky]

OK, total face palm moment. Not only was I wrong about replace-provider not working with TFC. (See here) but my supposed workaround was, most likely, due to my use of the replace-provider command.

Hopefully some day this will all be less mysterious to me.

[artikumari27]

I am facing similar issue while running terraform init.
error comes.:

• Finding latest version of hashicorp/null...
• Finding hashicorp/aws versions matching "3.15.0"...
• Installing hashicorp/archive v2.0.0...
• Installing hashicorp/null v3.0.0...
• Installed hashicorp/null v3.0.0 (signed by HashiCorp)
• Installing hashicorp/aws v3.15.0...
• Installed hashicorp/aws v3.15.0 (signed by HashiCorp)

Error: Failed to install provider

Error while installing hashicorp/archive v2.0.0: open
.terraform\plugins\registry.terraform.io\hashicorp\archive\2.0.0\windows_amd64\terraform-provider-archive_v2.0.0_x5.exe:
The system cannot find the path specified.

[artikumari27]

facing similar kind of issue while running terraform init command.
terraform {
required_version = "0.13.2"

backend "remote" {
hostname = "terraform.xxxx.com"
organization = "xxxxxxxx"

workspaces {
  name = "xxxxxx"
}

}

required_providers {
aws = {
source = "hashicorp/aws"
version = "3.15.0"
}
}
}

provider "aws" {}

Error

• Finding latest version of hashicorp/archive...
• Finding hashicorp/aws versions matching "3.15.0"...
• Installing hashicorp/archive v2.0.0...
• Installing hashicorp/null v3.0.0...
• Installed hashicorp/null v3.0.0 (signed by HashiCorp)
• Installing hashicorp/aws v3.15.0...
• Installed hashicorp/aws v3.15.0 (signed by HashiCorp)

Error: Failed to install provider

.terraform\plugins\registry.terraform.io\hashicorp\archive\2.0.0\windows_amd64\terraform-provider-archive_v2.0.0_x5.exe:
The system cannot find the path specified.

[agaffney]

I think I've managed to figure out what's going on here. This can only happen when using workspaces. When I cleared my .terraform dir and ran terraform init, it was on the default workspace, which has an empty state. When I switched to the desired workspace and ran terraform init again, it now downloads the compatibility -/whatever providers, and then everything works.

[fl-max]

@agaffney Thanks, this worked for me.

Manually running the replace-provider command on each state/workspace was not a viable option for us so just adding an extra init step after selecting the workspace in our Jenkins pipelines allowed for an automated path forward.

[bte8444]

I can confirm that @agaffney's fix worked for me as well. I was having the exact same issue with the Azurerm provider. The double init did the trick. Thanks!