Terraform: Provider download service no longer returning Protocol Version header

Created on 12 Jul 2017 · 22Comments · Source: hashicorp/terraform

PROVIDER ISSUES

PLEASE NOTE: Terraform has split out the builtin Providers into their own repositories. For any Provider issues, please open all issues and pull requests in the corresponding repository. An index of supported Providers can be found here:

https://github.com/terraform-providers

All other issues (that appear to affect multiple or all providers) may be an issue with Terraform's core, and should be opened here.

Hi there,

Thank you for opening an issue. Please note that we try to keep the Terraform issue tracker reserved for bug reports and feature requests. For general usage questions, please see: https://www.terraform.io/community.html.

Terraform Version

0.10.0 beta 1. Also observed on 0.10.0 beta 2

Debug Output

https://gist.github.com/DevOpsFu/392a5de72df538776ff7d632e921d709

Expected Behavior

Providers should have been downloaded on the fly

Actual Behavior

All plugin downloads failed with an error like:

no versions of "consul" compatible with the plugin ProtocolVersion

Steps to Reproduce

Any terraform plan that requires a download of a plugin from the plugin service.

Important Factoids

This appears to have started happening between about 2300 on 11th July and 0700 on 12th of July, UTC.

Manually curling the plugin service shows that the expected X-TERRAFORM_PROTOCOL_VERSION header is not being returned

bug core

Source

DevOpsFu

👍3

All 22 comments

Hi same for all plugins:

Initializing provider plugins...

Downloading plugin for provider "aws"...

No available provider "aws" plugins are compatible with this Terraform version.

From time to time, new Terraform major releases can change the requirements for
plugins such that older plugins become incompatible.

Terraform checked all of the plugin versions matching the given constraint:
(any version)

Unfortunately, none of the suitable versions are compatible with this version
of Terraform. If you have recently upgraded Terraform, it may be necessary to
move to a newer major release of this provider. Alternatively, if you are
attempting to upgrade the provider to a new major version you may need to
also upgrade Terraform to support the new version.

Consult the documentation for this provider for more information on
compatibility between provider versions and Terraform versions.

Downloading plugin for provider "vault"...

No available provider "vault" plugins are compatible with this Terraform version.

From time to time, new Terraform major releases can change the requirements for
plugins such that older plugins become incompatible.

Terraform checked all of the plugin versions matching the given constraint:
(any version)

Consult the documentation for this provider for more information on
compatibility between provider versions and Terraform versions.

Downloading plugin for provider "dyn"...

No available provider "dyn" plugins are compatible with this Terraform version.

From time to time, new Terraform major releases can change the requirements for
plugins such that older plugins become incompatible.

Terraform checked all of the plugin versions matching the given constraint:
(any version)

Consult the documentation for this provider for more information on
compatibility between provider versions and Terraform versions.

error: None
Plugin reinitialization required. Please run "terraform init".

Reason: Could not satisfy plugin requirements.

Plugins are external binaries that Terraform uses to access and manipulate

resources. The configuration provided requires plugins which can't be located,

don't satisfy the version constraints, or are otherwise incompatible.

3 error(s) occurred:

provider.vault: no suitable version installed

version requirements: "(any version)"

versions installed: none
provider.dyn: no suitable version installed

version requirements: "(any version)"

versions installed: none
provider.aws: no suitable version installed

version requirements: "(any version)"

versions installed: none

Terraform automatically discovers provider requirements from your

configuration, including providers used in child modules. To see the

requirements and constraints from each module, run "terraform providers".

Any ETA on this bug ?

liorco250 on 12 Jul 2017

Maybe @jbardin can help escalate?

coreypobrien on 12 Jul 2017

Hi,

Sorry about the inconvenience here, I'm not sure what happened yet to cause those headers to be removed.

Since the protocol version had to be moved to a header for the releases site, I think we should probably make the absence of the header non-fatal. While it may cause confusion later if the protocol version doesn't match, it will at least make an attempt to in the common case.

In the meantime for anyone hitting this while using the beta, you can download the provider manually from https://releases.hashicorp.com/, and simply unpack it in your config directory to run terraform.

@grubernaut @sethvargo, Any idea what happened to the X-TERRAFORM_PROTOCOL_VERSION header?

jbardin on 12 Jul 2017

We too ran into this issue. In our case, we had to drop the binary from the provider we were using (fastly) as well as this provider binary: https://releases.hashicorp.com/terraform-provider-terraform/0.1.0/

udryan10 on 12 Jul 2017

Hi all, sorry for this issue!

This is currently being worked on, and should be fixed soon. Thanks!

grubernaut on 12 Jul 2017

Hi all,

Sorry about this. This should be fixed now. Could you please verify?

sethvargo on 12 Jul 2017

We are still seeing this failure on v0.0.10-beta1.

barberta on 12 Jul 2017

Hey @barberta

What provider are you trying to download and for which platform?

sethvargo on 12 Jul 2017

The archive provider specifically isn't working for me, but the others are now:

Initializing provider plugins...
- Downloading plugin for provider "local"...
- Downloading plugin for provider "tls"...
- Downloading plugin for provider "aws"...
- Downloading plugin for provider "archive"...

No available provider "archive" plugins are compatible with this Terraform version.

From time to time, new Terraform major releases can change the requirements for
plugins such that older plugins become incompatible.

Terraform checked all of the plugin versions matching the given constraint:
    (any version)

Unfortunately, none of the suitable versions are compatible with this version
of Terraform. If you have recently upgraded Terraform, it may be necessary to
move to a newer major release of this provider. Alternatively, if you are
attempting to upgrade the provider to a new major version you may need to
also upgrade Terraform to support the new version.

Consult the documentation for this provider for more information on
compatibility between provider versions and Terraform versions.

- Downloading plugin for provider "template"...
- Downloading plugin for provider "null"...
- Downloading plugin for provider "ignition"...
- Downloading plugin for provider "random"...

With debug: 2017/07/12 14:48:19 [WARNING] missing x-terraform-protocol-version from: https://releases.hashicorp.com/terraform-provider-archive/0.1.0/terraform-provider-archive_0.1.0_darwin_amd64.zip?checksum=sha256:9b25b0b331f58a8ce212317147a30e76e7560f4aa4f8914aa8ee232c04b7bf4b

coreypobrien on 12 Jul 2017

@sethvargo the issue @udryan10 and I are seeing is specifically regarding the terraform-provider-terraform provider. Manually downloading and extracting this seems to fix our issue as identified earlier.

barberta on 12 Jul 2017

Below is our output from running terraform init:

```Initializing the backend...

Initializing provider plugins...

Downloading plugin for provider "terraform"...

No available provider "terraform" plugins are compatible with this Terraform version.

From time to time, new Terraform major releases can change the requirements for
plugins such that older plugins become incompatible.

Terraform checked all of the plugin versions matching the given constraint:
(any version)

Consult the documentation for this provider for more information on
compatibility between provider versions and Terraform versions.```

barberta on 12 Jul 2017

Hey @barberta

Thank you for working with us on this. I'm really sorry about the inconvenience. Could you run this and share the response headers:

curl -svo /dev/null https://releases.hashicorp.com/terraform-provider-terraform/0.1.0/terraform-provider-terraform_0.1.0_linux_amd64.zip

I'm wondering if Terraform has a cached copy of the binary without the header. @jbardin can you chime in a bit about how the provider download works?

sethvargo on 12 Jul 2017

Not a problem @sethvargo. Response headers are as follows:
< Cache-Control: max-age=31536000, stale-white-revalidate=86400, stale-if-error=604800, public < Content-Disposition: attachment < Last-Modified: Wed, 12 Jul 2017 17:52:40 GMT < ETag: "08ec51eb00164ba6b8e0c3d0a5452df5" < Content-Type: application/zip < Strict-Transport-Security: max-age=31536000; includeSubDomains; preload < X-XSS-Protection: 1; mode=block < X-Content-Type-Options: nosniff < X-Frame-Options: sameorigin < Content-Length: 15698006 < Accept-Ranges: bytes < Date: Wed, 12 Jul 2017 19:23:23 GMT < Connection: keep-alive

barberta on 12 Jul 2017

Okay @barberta

I think I narrowed it down. Can you try that curl command again please? You should see an x-terraform-protocol-version in the response headers somewhere.

sethvargo on 12 Jul 2017

@sethvargo i see it there now.
< Cache-Control: max-age=31536000, stale-white-revalidate=86400, stale-if-error=604800, public < Content-Disposition: attachment < Last-Modified: Wed, 12 Jul 2017 19:59:36 GMT < ETag: "08ec51eb00164ba6b8e0c3d0a5452df5" < Content-Type: application/zip < Strict-Transport-Security: max-age=31536000; includeSubDomains; preload < x-terraform-protocol-version: 4 < X-XSS-Protection: 1; mode=block < X-Content-Type-Options: nosniff < X-Frame-Options: sameorigin < Content-Length: 15698006 < Accept-Ranges: bytes < Date: Wed, 12 Jul 2017 20:01:41 GMT < Connection: keep-alive

barberta on 12 Jul 2017

And that seems to have solved the issue at least from our perspective. Thanks again for the quick resolution.

```Initializing the backend...

Initializing provider plugins...

Downloading plugin for provider "terraform"...

The following providers do not have any version constraints in configuration,
so the latest version was installed.

To prevent automatic upgrades to new major versions that may contain breaking
changes, it is recommended to add version = "..." constraints to the
corresponding provider blocks in configuration, with the constraint strings
suggested below.

provider.terraform: version = "~> 0.1"

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.```

barberta on 12 Jul 2017

👍

lmayorga1980 on 3 Aug 2017

@liorco250

Did you get your problem fixed?

I still have the same issue.

No available provider "aws" plugins are compatible with this Terraform version.

From time to time, new Terraform major releases can change the requirements for
plugins such that older plugins become incompatible.

Terraform checked all of the plugin versions matching the given constraint:
    (any version)

version

$ terraform12 -version
Terraform v0.12.0-beta1

ozbillwang on 2 Apr 2019

Hi @ozbillwang ,
The issue you are experiencing is not related to this one, but I can help! The terraform 0.12-beta release package included compatible provider binaries, which you can move to the plugins directory.
There is some more basic information about provider compatibility in the 0.12 beta release blog post.

mildwonkey on 2 Apr 2019

👍1

@mildwonkey

Thanks, but I still don't make it work.

I downloaded the package, and unzip it, there is only one binary terraform

-rwxrwxr-x@ 1 bill  staff  46218088  1 Mar 08:12 terraform
-rw-r--r--@ 1 bill  staff  14309302  3 Apr 11:11 terraform_0.12.0-beta1_darwin_amd64.zip

I also clean the folder ~/.terraform.d , local .terraform, run terraform12 (version of 0.12 beta1), still get the same error.

I went through 0.12 beta release blog post already, still not sure how to make it work. Any hints for me?

Updates

I got what need be done

you can try installing the provided development snapshots as third-party providers:

The url is: http://terraform-0.12.0-dev-snapshots.s3-website-us-west-2.amazonaws.com/

ozbillwang on 3 Apr 2019

I am so sorry for that @ozbillwang , I was thinking of the alpha releases which we bundled with the providers. That was my mistake. Thank you for updating this for me and any one else that runs across the same issue!

mildwonkey on 3 Apr 2019

👍1

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.