Terraform: Internalize all of the Go packages in this repository

[apparentlymart]

Now that the SDK has moved out into a separate repository, this main Terraform repository is a _main_ Go module rather than a _library_ Go module, and so none of the Go package APIs in here are intended for external consumption and they are not subject to semver-style backward compatibility promises.

Terraform is an older codebase that predates the introduction of the internal mechanism for to the Go toolchain that packages are not for external use, but since that is now the expected way to communicate that we're going to retroactively move all of Terraform's Go packages (aside from package main in the root) down under the internal directory.

We're intending to do this prior to tagging a Terraform 1.0 in order to avoid making it appear to the Go toolchain that our v1.0.0 tag is a stable release of a Go Module containing library packages. However, we will have some delay before doing it in order to give a chance to land some already-opened pull requests that would likely end up with merge conflicts once the top-level packages have new import paths.

[wata727]

I'm so interested in this. As far as I know, some third-party utilities (including TFLint) are importing packages from this repository. And some of the packages are not included in terraform-plugin-sdk. For example, TFLint uses an internal API to load configuration files and evaluate expressions.

Are you planning to offer such functionality as a separate repository? If don't, I'm considering writing a Go module like "libterraform", just like when I opened https://github.com/hashicorp/terraform/issues/17904.

I agree with this change. I would like to discuss the policy of future third-party products. Thanks to the Terraform team.

[apparentlymart]

Hi @wata727,

The supported public interfaces for Terraform are protocols and file formats rather than in-process Go function calls, so I'd recommend integrating via external implementations of the documented protocols and formats. Because Terraform is open source, it's fine to use copies of code in this repository as the basis of those implementations, but it's important to distinguish between the documented interfaces and the current implementation details if you are building something that should work with multiple versions of Terraform.

Part of the 1.0 release will be some stronger promises about which parts of Terraform's behavior are integration points you can rely on.