Terraform: Terraform 0.13 downloading wrong version of provider

[nathankodilla]

Terraform Version

0.13.0

Terraform Configuration Files

terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
      version = "2.62.0"
    }
  }
  required_version = ">= 0.13"
}

terraform {
  backend "s3" {
    bucket = "xxxxxxxxxxxxxxxxxx"
    key    = "xxxxxxxxxxxxxx"
    region = "us-east-1"
    dynamodb_table = "xxxxxxxxxxxxxxxxxxx"
    role_arn = "arn:aws:iam::xxxxxxxxxxx"
  }
}

provider "aws" {
  allowed_account_ids = ["xxxxxxxxxx"]
  assume_role {
    role_arn = "arn:aws:iam::xxxxxxxxxx"
  }
  region     = "us-east-1"
  version = "~> 2.62.0"
}

Expected Behavior

Terraform will only download version 2.62.0 of aws provider.

Actual Behavior

Terraform downloaded version 2.62.0 and 3.1.0 of aws provider.

- terraform.io/builtin/terraform is built in to Terraform
- Finding latest version of -/aws...
- Finding hashicorp/aws versions matching "~> 2.70.0"...
- Installing hashicorp/aws v2.62.0...
- Installed hashicorp/aws v2.62.0 (signed by HashiCorp)
- Installing -/aws v3.1.0...
- Installed -/aws v3.1.0 (signed by HashiCorp)

The following providers do not have any version constraints in configuration,
so the latest version was installed.

To prevent automatic upgrades to new major versions that may contain breaking
changes, we recommend adding version constraints in a required_providers block
in your configuration, with the constraint strings suggested below.

* -/aws: version = "~> 3.1.0"

This then causes an issue when I run terraform apply where it shows it can't decode a field (issue described here: https://github.com/hashicorp/terraform/issues/25752).

Steps to Reproduce

1. terraform 0.13upgrade
2. terraform init

[Luwdo]

Without the required_providers version set call it does only download the 3.1.0 version, but this would defiantly break if you plan to use an older provider with 0.13.0. It is possible that the 2.62 provider is not compatible with 0.13.0 and its not marked as an incompatibility in the provider dependency tree.

Is it possible to confirm that the 3.1 provider is actually being used with terraform calls even when your version constraint should prevent that?

[brenak]

Removing my comments, the proper fix is listed below by alisdair and kstromeiraos

[kstromeiraos]

This command fixed it for me:

terraform state replace-provider -- -/aws hashicorp/aws

[alisdair]

Explanation & Workaround

The source of the -/aws provider qualified name is your state file. You can verify this by running terraform providers.

Running terraform apply (with a no-change plan) should upgrade the state file, leaving you with only the hashicorp/aws provider required. Please give that a try.

If for some reason you cannot run terraform apply, you can update the providers in your state file using terraform state replace-provider -- -/aws hashicorp/aws.

All that said, this is not ideal behaviour, and if we could fix it by somehow merging the provider requirements for legacy providers before attempting installation, I think that would be a better user experience.

Reproduction

1. Create main.tf:

   provider "null" {
     version = "2.1.0"
   }
   
   resource "null_resource" "none" { }
   

2. terraform-0.12.29 init && terraform-0.12.29 apply -auto-approve
3. terraform-0.13.0 init

• Expected: only version 2.1.0 of null provider is installed
• Actual: both 2.1.0 and 2.1.2 are installed

[akindo]

Explanation & Workaround

The source of the -/aws provider qualified name is your state file. You can verify this by running terraform providers.

Running terraform apply (with a no-change plan) should upgrade the state file, leaving you with only the hashicorp/aws provider required. Please give that a try.

If for some reason you cannot run terraform apply, you can update the providers in your state file using terraform state replace-provider -- -/aws hashicorp/aws.

All that said, this is not ideal behaviour, and if we could fix it by somehow merging the provider requirements for legacy providers before attempting installation, I think that would be a better user experience.

Reproduction

1. Create main.tf:
   ```terraform
   provider "null" {
     version = "2.1.0"
   }

   resource "null_resource" "none" { }
   ```

2. `terraform-0.12.29 init && terraform-0.12.29 apply -auto-approve`

3. `terraform-0.13.0 init`


* Expected: only version 2.1.0 of null provider is installed

* Actual: both 2.1.0 and 2.1.2 are installed

Yes, I believe some of us can't run terraform apply after upgrading to v. 0.13 and running terraform 0.13upgrade, as our code isn't compatible with the latest version (3.2.0) of the AWS provider (https://github.com/hashicorp/terraform/issues/25752), which the state is using. E. g. I got

$ terraform apply                                                     

Error: Invalid resource instance data in state

  on ../../modules/session-manager/main.tf line 63:
  63: resource "aws_iam_instance_profile" "session_manager_ec2" {

Instance module.session-manager.aws_iam_instance_profile.session_manager_ec2
data could not be decoded from the state: unsupported attribute "roles".

One doesn't expect that the code must be compatible with AWS provider v. 3.2.0 when the code is still specifying v. 2.x. Perhaps make terraform state replace-provider -- -/aws hashicorp/aws part of terraform 0.13upgrade?

[remilapeyre]

Perhaps make terraform state replace-provider -- -/aws hashicorp/aws part of terraform 0.13upgrade?

I don't think this is possible because the terraform 0.13upgrade changes the configuration code and does not modify the remote state. You will have to run it once per project. On the other hand, if you have multiple deployments of your project, e.g. when using Terraform workspaces, you will have to replace the providers in each of them, running terraform state replace-provider -- -/aws hashicorp/aws for each workspace.

[mohsinhijazee]

This command fixed it for me:

terraform state replace-provider -- -/aws hashicorp/aws

This is another of the issue that is happening and for me, this worked:

terraform state replace-provider registry.terraform.io/-/aws registry.terraform.io/hashicorp/aws
terraform state replace-provider registry.terraform.io/-/template registry.terraform.io/hashicorp/template

Again, this should be handled with terraform 0.13upgrade (which does only source code rewrite) or should be a subcommand of the state command.