Terraform: Cannot build Terraform because of git.apache.org downtime

[chreble]

Terraform v0.12.5

go: git.apache.org/[email protected]: git fetch -f https://git.apache.org/thrift.git refs/heads/*:refs/heads/* refs/tags/*:refs/tags/* in /tmp/terraform-20190902-32196-9gb7xn/terraform-0.12.7/pkg/mod/cache/vcs/83dba939f95a790e497d565fc4418400145a1a514f955fa052f662d56e920c3e: exit status 128:
        fatal: unable to access 'https://git.apache.org/thrift.git/': Failed to connect to git.apache.org port 443: Connection refused

Expected Behavior

It should build normally

Actual Behavior

It doesn't build

Steps to Reproduce

Build Terraform

I think the location of the thrift repository should be updated from git.apache.org/thrift.git to github.com/apache/thrift.

The issue has been listed on https://status.apache.org/ for several days.

[next-joaquinfernandezcampo]

@chreble you can get it to build adding this line to the go.mod:

replace git.apache.org/thrift.git => github.com/apache/thrift v0.0.0-20180902110319-2566ecd5d999

This works if you're building a provider as my case, you can probably run a command beforehand to force it to replace before building.

[chreble]

I'm not building Terraform from the source but using Homebrew. It's been days that git.apache.org is down so I think this fix is critical...

[apparentlymart]

The usual way to build Terraform is to use -mod=vendor on the Go build/install command line.

The official packages we distribute contain pre-built binaries, so building from source is not a primary usage pattern. We will fix the dependency in due course in order to unblock our own development, but if Homebrew is building its own binaries from source then would be best to make it use -mod=vendor to match the way the official binaries are built.

The Terraform Core team is not working today due to the US Labor Day holiday so unfortunately this won't be addressed on our end until tomorrow at least. If you are being blocked by this today, please consider using the official releases for the moment.

[chreble]

That's what I did, thanks @apparentlymart 👍

[next-joaquinfernandezcampo]

@apparentlymart This issue also affects provider development so I'm guessing it's going to start blocking some official providers soon.

[apparentlymart]

If you are working in a repository that doesn't already have that dependency vendored then you can also try using the Go public module proxy, and see if it has that library cached.

Protecting against this "vanishing dependency" problem is a key goal of that proxy, and it will be on by default in Go 1.13. You can optionally enable it in 0.12 though whether that dependency is cached there will depend on whether someone else previously enabled the proxy and then built a module that depended on that one, which I'm not sure about and can't test because I'm not at a computer right now.

[paulrowlands]

I'm just trying to install Terrafrom using brew; the error brought me here...

brew install terraform
==> Downloading https://github.com/hashicorp/terraform/archive/v0.12.7.tar.gz
Already downloaded: /home/paul/.cache/Homebrew/downloads/a646c327f735cb38a3a77bde973c81e05e9cc5d74daa00cddc0ad1cc8f1fb277--terraform-0.12.7.tar.gz
==> make tools bin
Last 15 lines from /home/paul/.cache/Homebrew/Logs/terraform/01.make:
go: finding github.com/golang/mock/mockgen latest
go: downloading github.com/golang/mock v1.3.1
go: extracting github.com/golang/mock v1.3.1
go: finding github.com/hashicorp/terraform/internal latest
go: finding github.com/hashicorp/terraform v0.12.7
go: downloading github.com/hashicorp/terraform v0.12.7
go: extracting github.com/hashicorp/terraform v0.12.7
go: finding git.apache.org/thrift.git v0.0.0-20180902110319-2566ecd5d999
go: git.apache.org/thrift.[email protected]: git fetch -f https://git.apache.org/thrift.git refs/heads/:refs/heads/ refs/tags/:refs/tags/ in /tmp/terraform-20190903-13712-1f1dnxh/terraform-0.12.7/pkg/mod/cache/vcs/83dba939f95a790e497d565fc4418400145a1a514f955fa052f662d56e920c3e: exit status 128:
fatal: unable to access 'https://git.apache.org/thrift.git/': Failed to connect to git.apache.org port 443: Connection timed out
go: error loading module requirements
2019/09/03 15:43:57 Loading input failed: exit status 1
plugin/mock_proto/generate.go:1: running "bash": exit status 1
Makefile:76: recipe for target 'generate' failed
make: * [generate] Error 1

READ THIS: https://docs.brew.sh/Troubleshooting

[pbartruff]

@chreble you can get it to build adding this line to the go.mod:

replace git.apache.org/thrift.git => github.com/apache/thrift v0.0.0-20180902110319-2566ecd5d999

This works if you're building a provider as my case, you can probably run a command beforehand to force it to replace before building.

I am trying to build the terraform-provider-aws and I am suffering from a connection refused too. Not sure if @next-joaquinfernandezcampo comment is a work-around, or I'm missing what to change.

[SimonBarendse]

If you are working in a repository that doesn't already have that dependency vendored then you can also try using the Go public module proxy, and see if it has that library cached.

Protecting against this "vanishing dependency" problem is a key goal of that proxy, and it will be on by default in Go 1.13. You can optionally enable it in 0.12 though whether that dependency is cached there will depend on whether someone else previously enabled the proxy and then built a module that depended on that one, which I'm not sure about and can't test because I'm not at a computer right now.

I can confirm that https://proxy.golang.org has the library cached. So setting GOPROXY=https://proxy.golang.org (or upgrading to go1.13) solves the issue.

[pbartruff]

@SimonBarendse Thanks for the tip. Go is not my strong suit (obviously) 😃.

[ankushchadha]

Both modules (github.com/apache/thrift and git.apache.org/thrift.git) are available in GoCenter. We do encourage setting GOPROXY in your builds. Here is the URL of the modules in question -

https://search.gocenter.io/github.com~2Fapache~2Fthrift/info

https://search.gocenter.io/git.apache.org~2Fthrift.git/info

[gcase555]

Its worth mentioning that the official documentation points people to pull the dependency from github, not the apache mirror https://thrift.apache.org/lib/go#using-thrift-with-go

[apparentlymart]

hashicorp/go-getter#204 should fix this, once merged and incorporated here. It eliminates the transitive dependency on this library altogether, since Terraform doesn't actually need it.

In the meantime, I suggest using GOFLAGS=-mod=vendor as a primary workaround (since that matches how Terraform's official releases are built anyway) or, if you don't wish to use the vendored dependencies for some reason, continuing to use GOPROXY if you need to access dependencies directly for some reason.

[apparentlymart]

Hi all!

22741 has upgraded to a newer version of go-getter which is in turn using newer versions of other dependencies such that the thrift library is no longer in the dependency graph at all, and thus the git.apache.org host should no longer be needed to fetch Terraform's dependencies. Therefore I'm going to close this out.

I would still suggest using -mod=vendor if you are building Terraform for "real" use, because that will then match as closely as possible how Terraform is built for its major releases, ensuring that you get exactly the dependencies given in this repository. Building against the upstreams of the same code should produce the same result, but it's redundant to re-fetch all of that upstream code when there's already a copy of it all in this repository anyway.

Thanks again for reporting this, and for your patience while we investigated the source of the problem to find the best path forward.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.