Terraform: Terraform Testing Framework

[bscott]

Any Ideas around testing Terraform, my group has a requirement to tag all AWS instances, it would be nice to assert that a AWS TAG is missing. Thinking RSpec for Terraform or like Test-Kitchen.

Thoughts?

[apparentlymart]

On my team we wrote a little "linting" tool that takes as input a Terraform state and then prints out a list of complaints about how things were created. I think this isn't quite the same thing as what you're asking about, since it's only testing what Terraform thinks was created rather than what actually exists, but our purpose is the same as the example you gave: to check whether our resource tagging scheme is being respected.

We didn't release the tool publicly because it just encodes a few rules that are very specific to our environment, but it's a pretty simple thing: parse the state file as JSON, and then walk over all of the resources and apply various rules depending on resource type.

[bscott]

@apparentlymart thanks thats a good idea and a good place to start.

[bscott]

I created https://github.com/bscott/terralint to get something created and see what the community can come up with!

[radeksimko]

I like the idea of parsing tfstate files and making some conclusions based on it.

I think the challenge is to be able to test changes to an existing infrastructure. Testing fresh, new directory with no previous state IMO isn't that difficult and if you're not concerned too much about the costs & time, you could do this per pull request.

However testing incremental changes with an existing state isn't trivial, because you need to have clear "rollback" plans so that after you run such test, the infrastructure remains exactly in the same state as it was before.

[radeksimko]

btw. if you only care about AWS, there are some services you may leverage:

• https://aws.amazon.com/inspector/
• https://aws.amazon.com/config/

[apparentlymart]

@radeksimko being able to apply this sort of "linting" approach to Terraform _plans_ was the main motivation for what I was doing over in #4610. As I noted over there, it could be used to detect and automatically apply "trivial" changes, it could be used to completely _block_ other sorts of changes that are known to be incorrect/harmful.

Of course linting a plan can't catch all errors that would arise when applying the plan, so it doesn't quite get to what you were talking about, but there are several "common" mistakes I've seen in our infrastructure that I'd like to catch before they are live, and they're specific enough to our particular configuration that it doesn't feel right to put them in core Terraform.

[ncs-alane]

Hey all:

I'm excited to annouce kitchen-terraform. This gem is a group of Test Kitchen plugins that allows for integration testing of Terraform configuration using Inspec. Please check it out and feel free to contribute!

[stack72]

Hi @bscott

I am going to go ahead and close this one out. This has spawned some great links :) Thanks for opening it

Paul

[walterdolce]

Hi all,

I'm adding this here as I think it's a valuable tool (and the only one I could find out there) which allows people to "_unit test_"* terraform's AWS resources. It's young and surely needs love/attention. Still adds value when it comes to testing Terraform & AWS.

https://github.com/k1LoW/awspec

• I used quotes there because I believe that is not _really_ unit testing. Unit testing assumes the ability to "dry run" or mock things out (something chefspec does for Chef resources). Rather, with awspec you make sure the AWS resources managed by Terraform are in the expected state.

[pradeepbhadani]

I used awspec in past and found it very useful to test aws resources.

[timurb]

I've been using plain RSpec with https://github.com/tobyclemson/ruby_terraform for running Terraform and testing against outputs -- works fine for integration-like tests.

For unit-like tests I had to write a library though for parsing output of terraform plan.
See https://github.com/hashicorp/terraform/issues/13738 for possible improvement around this.

[alexharv074]

The design of Terraform is similar to the design of Puppet, and the Terraform _plan_ is analogous to Puppet's _catalog_, therefore it seems to me that the Terraform equivalent of Rspec-puppet is missing from the Terraform ecosystem and should be written as soon as possible. I agree with @radeksimko that testing state changes is a harder problem, just as it is in Puppet, but it hardly changes the fact that an official Rspec-puppet equivalent would be incredibly useful.

[gpeden]

I have spent some time learning infrastructure testing with kitchen, terraform, awspec. I put together a training pack here https://github.com/gpeden/awspec-kitchen-terraform . maybe someone will find it useful.

[brikis98]

We recently open sourced Terratest, our swiss army knife for testing infrastructure code.

Today, you're probably testing all your infrastructure code manually by deploying, validating, and undeploying. Terratest helps you automate this process:

1. Write tests in Go.
2. Use helpers in Terratest to execute your real IaC tools (e.g., Terraform, Packer, etc.) to deploy real infrastructure (e.g., servers) in a real environment (e.g., AWS).
3. Use helpers in Terratest to validate that the infrastructure works correctly in that environment by making HTTP requests, API calls, SSH connections, etc.
4. Use helpers in Terratest to undeploy everything at the end of the test.

Here's an example test for some Terraform code:

terraformOptions := &terraform.Options {
  // The path to where your Terraform code is located
  TerraformDir: "../examples/terraform-basic-example",
}

// This will run `terraform init` and `terraform apply` and fail the test if there are any errors
terraform.InitAndApply(t, terraformOptions)

// At the end of the test, run `terraform destroy` to clean up any resources that were created
defer terraform.Destroy(t, terraformOptions)

// Run `terraform output` to get the value of an output variable
instanceUrl := terraform.Output(t, terraformOptions, "instance_url")

// Verify that we get back a 200 OK with the expected text
// It can take a minute or so for the Instance to boot up, so retry a few times
expected := "Hello, World"
maxRetries := 15
timeBetweenRetries := 5 * time.Second
http_helper.HttpGetWithRetry(t, instanceUrl, 200, expected, maxRetries, timeBetweenRetries)

These are integration tests, and depending on what you're testing, can take 5 - 50 minutes. It's not fast (though using Docker and test stages, you can speed some things up), and you'll have to work to make the tests reliable, but it is well worth the time. Our small team has been able to use Terratest to maintain an Infrastructure as Code Library which has over 250,000 lines of Terraform/Packer/Docker/Go/Python etc code and is used in production by hundreds of companies. Without tests, there's no way we could maintain this code.

Check out the Terratest repo for docs and lots of examples of various types of infrastructure code and the corresponding tests for them.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.