Terraform: "Error: Failed to query available provider packages" with in-house provider

[wolviecb]

I've seen to be stuck in this error here with a local provider I've built. When I try to init with terraform 0.13 I get an Error: Failed to query available provider packages. I'd started reporting this on #25218 and @alisdair pointed that this is probably another issue and asked to another report

Terraform Version

0.13.0 beta2

Terraform Configuration Files

terraform {
  required_version = ">= 0.13"
  required_providers {
    tranzaxis = {
      source  = "local/txr/tranzaxis"
      version = "0.0.1"
    }
  }
}


resource "tranzaxis_instance_state" "instance_1" {
  environment    = "test-env"
  subsystem      = "something"
  tx_instance_id = "1"
  address        = "hostname"
  aws_id         = "i-1234567890"
}

Debug Output

$ terraform init -plugin-dir=terraform.d
2020/06/23 12:57:05 [INFO] Terraform version: 0.13.0 beta2
2020/06/23 12:57:05 [INFO] Go runtime version: go1.14.2
2020/06/23 12:57:05 [INFO] CLI args: []string{"/Users/thomas.andrade/txr/repos/tft/terraform-provider-tranzaxis/test/terraform", "init", "-plugin-dir=terraform.d"}
2020/06/23 12:57:05 [DEBUG] Attempting to open CLI config file: /Users/thomas.andrade/.terraformrc
2020/06/23 12:57:05 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2020/06/23 12:57:05 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2020/06/23 12:57:05 [DEBUG] ignoring non-existing provider search directory /Users/thomas.andrade/.terraform.d/plugins
2020/06/23 12:57:05 [DEBUG] ignoring non-existing provider search directory /Users/thomas.andrade/Library/Application Support/io.terraform/plugins
2020/06/23 12:57:05 [DEBUG] ignoring non-existing provider search directory /Library/Application Support/io.terraform/plugins
2020/06/23 12:57:05 [INFO] CLI command args: []string{"init", "-plugin-dir=terraform.d"}

Initializing the backend...
2020/06/23 12:57:05 [TRACE] Meta.Backend: no config given or present on disk, so returning nil config
2020/06/23 12:57:05 [TRACE] Meta.Backend: backend has not previously been initialized in this working directory
2020/06/23 12:57:05 [DEBUG] New state was assigned lineage "c4ad9d37-7a52-c2da-816b-bb1dd98483c9"
2020/06/23 12:57:05 [TRACE] Meta.Backend: using default local state only (no backend configuration, and no existing initialized backend)
2020/06/23 12:57:05 [TRACE] Meta.Backend: instantiated backend of type <nil>
2020/06/23 12:57:05 [DEBUG] checking for provisioner in "terraform.d"
2020/06/23 12:57:05 [INFO] Failed to read plugin lock file .terraform/plugins/darwin_amd64/lock.json: open .terraform/plugins/darwin_amd64/lock.json: no such file or directory
2020/06/23 12:57:05 [TRACE] Meta.Backend: backend <nil> does not support operations, so wrapping it in a local backend
2020/06/23 12:57:05 [TRACE] backend/local: state manager for workspace "default" will:
 - read initial snapshot from terraform.tfstate
 - write new snapshots to terraform.tfstate
 - create any backup at terraform.tfstate.backup

Initializing provider plugins...
2020/06/23 12:57:05 [TRACE] statemgr.Filesystem: reading initial snapshot from terraform.tfstate
- Finding local/txr/tranzaxis versions matching "0.0.1"...
2020/06/23 12:57:05 [TRACE] statemgr.Filesystem: read snapshot with lineage "f891f123-285a-6782-bf96-67c90c011f40" serial 10
2020/06/23 12:57:05 [DEBUG] init: overriding provider plugin search paths
2020/06/23 12:57:05 [DEBUG] will search for provider plugins in [terraform.d]
2020/06/23 12:57:05 [TRACE] providercache.fillMetaCache: scanning directory .terraform/plugins
2020/06/23 12:57:05 [TRACE] providercache.fillMetaCache: error while scanning directory .terraform/plugins: cannot search .terraform/plugins: lstat .terraform/plugins: no such file or directory
2020/06/23 12:57:05 [WARN] Failed to scan provider cache directory .terraform/plugins: cannot search .terraform/plugins: lstat .terraform/plugins: no such file or directory
2020/06/23 12:57:05 [TRACE] getproviders.SearchLocalDirectory: found local/txr/tranzaxis v0.0.1 for darwin_amd64 at terraform.d/local/txr/tranzaxis/0.0.1/darwin_amd64
2020/06/23 12:57:05 [TRACE] providercache.Dir.InstallPackage: installing local/txr/tranzaxis v0.0.1 from terraform.d/local/txr/tranzaxis/0.0.1/darwin_amd64
2020/06/23 12:57:05 [TRACE] providercache.fillMetaCache: scanning directory .terraform/plugins
2020/06/23 12:57:05 [TRACE] getproviders.SearchLocalDirectory: found local/txr/tranzaxis v0.0.1 for darwin_amd64 at .terraform/plugins/local/txr/tranzaxis/0.0.1/darwin_amd64
2020/06/23 12:57:05 [TRACE] providercache.fillMetaCache: including .terraform/plugins/local/txr/tranzaxis/0.0.1/darwin_amd64 as a candidate package for local/txr/tranzaxis 0.0.1
2020/06/23 12:57:05 [TRACE] providercache.fillMetaCache: using cached result from previous scan of .terraform/plugins
- Finding latest version of -/tranzaxis...
- Installing local/txr/tranzaxis v0.0.1...
- Installed local/txr/tranzaxis v0.0.1 (unauthenticated)

Error: Failed to query available provider packages

Could not retrieve the list of available versions for provider -/tranzaxis:
provider registry.terraform.io/-/tranzaxis was not found in any of the search
locations

- terraform.d

Expected Behavior

terraform init succeeds

Actual Behavior

Terraform tries to download the provider from registry.terraform.io

[...]
- Finding latest version of -/tranzaxis...

Error: Failed to query available provider packages

Could not retrieve the list of available versions for provider -/tranzaxis:
provider registry.terraform.io/-/tranzaxis was not found in any of the search
locations

- terraform.d

It's reading some provider info from the state

$ terraform providers

Providers required by configuration:
.
└── provider[local/txr/tranzaxis] 0.0.1

Providers required by state:

    provider[registry.terraform.io/-/tranzaxis]

although I'm using a local provider

I've tried to init on a clean state and it works as expected, the applied resource has a provider reference that points to the local source that the old state doesn't

old state:

{
  [...]
 "resources": [
    {
      "mode": "managed",
      "type": "tranzaxis_instance_state",
      "name": "instance_1",
      "provider": "provider.tranzaxis",
      "instances": [...]
    }
 ]
}

new state:

{
  [...]
 "resources": [
    {
      "mode": "managed",
      "type": "tranzaxis_instance_state",
      "name": "instance_1",
      "provider": "provider[\"registry.klarna.com/txr/tranzaxis\"]",
      "instances": [...]
    }
 ]
}

Steps to Reproduce

1. terraform init

[alisdair]

@wolviecb Thanks for this report! I was able to reproduce this locally.

You should be able to work around this issue for now by using the terraform state replace-providers command to update your state to point to your new provider address. The command should be something like:

terraform state replace-provider -- -/tranzaxis registry.klarna.com/txr/tranzaxis

The output should look something like:

Terraform will perform the following actions:

  ~ Updating provider:
    - registry.terraform.io/-/tranzaxis
    + registry.klarna.com/txr/tranzaxis

Changing 1 resources:

  tranzaxis_instance_state. instance_1

Once this is done, terraform init should succeed and so should plan, apply, and so on. Please let me know whether this works for you!

I'm leaving this issue open, as we will address this problem with changes to documentation and possibly code.

[wolviecb]

The workaround fixed the issue! Thx!

[alisdair]

We've improved the 0.13 upgrade guide to note that this workaround is necessary.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.