Terraform: Terraform v0.10.0 how to use custom build to override official provider?

[ctavan]

Terraform Version

Terraform v0.10.0

Terraform Configuration Files

N/A

Debug Output

N/A

Panic Output

N/A

Expected Behavior

In Terraform 0.9.x it was easily possible to use a custom build of an official provider by simply putting the corresponding provider-binary into the PATH next to the terraform binary, e.g. I could simply build a custom terraform-provider-google, put it into the PATH and use it in combination with the official terraform release. Terraform would simply pick up the overridden binary from the path and use it for all google provider resources.

Actual Behavior

With terraform 0.10.0 all official plugins are downloaded to .terraform/plugins/darwin_amd64/ in the local working directory during terraform init.

I can successfully use unofficial providers (like the acme provider) by putting the provider binaries to ~/.terraform.d/plugins/terraform-provider-acme as described in #15705.

However this does not work for official providers. Putting my custom built provider binary to ~/.terraform.d/plugins/terraform-provider-google has no effect.

Following the documentation at https://www.terraform.io/docs/plugins/provider.html#provider-plugin-codebases did not help either: Even directly calling $GOPATH/bin/terraform (while $GOPATH/bin/terraform-provider-google is present) won't pick up the provider from $GOPATH/bin but will instead use the provider from .terraform/plugins/darwin_amd64 which was downloaded during terraform init.

What's the recommended way to use a custom build of an official terraform-provider?

Edit: What I have found as a workaround so far is to put _all_ plugins into a dedicated directory and then run terraform init -plugin-dir path/to/plugins (as described in https://www.terraform.io/docs/commands/init.html#plugin-installation). I still believe it would be nice though, if I could rely on automatic plugin download for everything that I don't have to touch and only manage those plugins on my own, that are either not official or that I have modified.

Steps to Reproduce

N/A

Important Factoids

N/A

References

• GH-15705

[jbardin]

Hi @ctavan,

Sorry this is still confusing, we do need to improve some of the documentation around some of the less common use cases.

The recommended path to supply vendored plugins is a terraform.d/plugins/os_arch/ directory in the current configuration directory. Currently the plugin will also be picked up if it's found in the root of the current config directory, or the directory containing the terraform binary. If there is a versioned provider found, in the .terraform/plugins directory or elsewhere, that will be preferred over the non-versioned file (which essentially has a version of 0.0.0), so you need to remove the the .terraform/ directory and run init again.

The .terraform/ directory isn't meant to be manipulated by the user, so it might make sense for init to have an option to completely replace all plugins in the future.

[ctavan]

@jbardin thanks for the clarification! Providing the plugins in terraform.d/plugins works perfectly fine for me.

I've proposed an improvement to the docs in #15745 which I believe would have helped me in the first place (and would have avoided the need for this clarifying github issue).

[jzampieron]

What is the recommend method to do this in v0.11+?

The suggestions here always result in the versioned binary being pulled from the remote site, as opposed to the unversioned local override.

I managed to get it to use the local version by putting a new, fake version number in my own executable. This isn't an ideal development flow for fast iteration. What other options are there?

The path terraform.d/plugins/darwin_amd64 was the appropriate path on Mac OS.

[jbardin]

@jzampieron,

The discovery paths haven't changed. Can you show an example of the configuration used and the exact location of the provider?

[Dominik-K]

@jbardin Happened to me also with v0.11.3. Putting the binary in all the search paths mentioned here didn't work.

It worked with removing the version constraint in the provider "aws" config:

2018/03/01 19:12:18 [DEBUG] found valid plugin: "aws", "0.0.0", "/Users/dominik/go/bin/terraform-provider-aws"

[apparentlymart]

Hi @Dominik-K,

Terraform uses a file naming scheme to determine the version number for each plugin. You can still use version constraints if you ensure that the filename is of the same form as you'd see in the official release archives, like:

terraform-provider-aws-v1.10.0

This will then be used _if_ the version number used matches the given version constraint.

We treat binaries without version numbers as "unversioned", mainly for compatibility with plugins that were built prior to the version mechanism being implemented. Version "0.0.0" is how _unversioned_ is represented.

We did not make any intentional change to the behavior of this since the 0.10 release, so if you've seen behavior change after upgrading from a 0.10 release to a 0.11 release then I'd be curious to hear more about that, if you'd be willing to open a new top-level issue for it so we can capture all the relevant information to help understand what's going on.

[binarylogic]

Agree that this process is cumbersome and confusing. To help others in the future:

Version "0.0.0" is how unversioned is represented.

This is not correct unless you actually specify a version of 0.0.0 in your provider declaration. For example:

provider "custom_provider" { ... }

Requires a binary to be placed in terraform.d/plugins/darwin_amd64/terraform-provider-custom_provider (note the missing v0.0.0 suffix).

[tmccombs]

I tried putting a custom build of the aws provider in ~/.terraform.d/plugins/linux_amd64/terraform-provider-aws-v2.29.1 but terraform wasn't able to find it.

If I specifically require the version "2.29.1" I get an error No provider "aws" plugins meet the constraint "2.29.1"

[tmccombs]

maybe having a custom build of a hashicorp provider doesn't interact well with the provider cache?

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.