Opa: Terraform tutorial does not support TF v0.12

[kumar-shantanu]

Expected Behavior

Should show some score

Actual Behavior

Doesn't show any score

Steps to Reproduce the Problem

Run opa eval --data terraform.rego --input tfplan.json "data.terraform.analysis.score"

with terraform.rego copied from the OPA documentation and tf.json containing below json

{"format_version":"0.1","terraform_version":"0.12.1","planned_values":{"root_module":{"resources":[{"address":"aws_autoscaling_group.my_asg","mode":"managed","type":"aws_autoscaling_group","name":"my_asg","provider_name":"aws","schema_version":0,"values":{"availability_zones":["us-west-1a"],"desired_capacity":4,"enabled_metrics":null,"force_delete":true,"health_check_grace_period":300,"health_check_type":"ELB","initial_lifecycle_hook":[],"launch_configuration":"my_web_config","launch_template":[],"max_size":5,"metrics_granularity":"1Minute","min_elb_capacity":null,"min_size":1,"mixed_instances_policy":[],"name":"my_asg","name_prefix":null,"placement_group":null,"protect_from_scale_in":false,"suspended_processes":null,"tag":[],"tags":null,"termination_policies":null,"timeouts":null,"wait_for_capacity_timeout":"10m","wait_for_elb_capacity":null}},{"address":"aws_autoscaling_group.my_asg2","mode":"managed","type":"aws_autoscaling_group","name":"my_asg2","provider_name":"aws","schema_version":0,"values":{"availability_zones":["us-west-2a"],"desired_capacity":4,"enabled_metrics":null,"force_delete":true,"health_check_grace_period":300,"health_check_type":"ELB","initial_lifecycle_hook":[],"launch_configuration":"my_web_config","launch_template":[],"max_size":6,"metrics_granularity":"1Minute","min_elb_capacity":null,"min_size":1,"mixed_instances_policy":[],"name":"my_asg2","name_prefix":null,"placement_group":null,"protect_from_scale_in":false,"suspended_processes":null,"tag":[],"tags":null,"termination_policies":null,"timeouts":null,"wait_for_capacity_timeout":"10m","wait_for_elb_capacity":null}},{"address":"aws_autoscaling_group.my_asg3","mode":"managed","type":"aws_autoscaling_group","name":"my_asg3","provider_name":"aws","schema_version":0,"values":{"availability_zones":["us-west-2b"],"desired_capacity":4,"enabled_metrics":null,"force_delete":true,"health_check_grace_period":300,"health_check_type":"ELB","initial_lifecycle_hook":[],"launch_configuration":"my_web_config","launch_template":[],"max_size":7,"metrics_granularity":"1Minute","min_elb_capacity":null,"min_size":1,"mixed_instances_policy":[],"name":"my_asg3","name_prefix":null,"placement_group":null,"protect_from_scale_in":false,"suspended_processes":null,"tag":[],"tags":null,"termination_policies":null,"timeouts":null,"wait_for_capacity_timeout":"10m","wait_for_elb_capacity":null}},{"address":"aws_instance.web","mode":"managed","type":"aws_instance","name":"web","provider_name":"aws","schema_version":1,"values":{"ami":"ami-09b4b74c","credit_specification":[],"disable_api_termination":null,"ebs_optimized":null,"get_password_data":false,"iam_instance_profile":null,"instance_initiated_shutdown_behavior":null,"instance_type":"t2.micro","monitoring":null,"source_dest_check":true,"tags":null,"timeouts":null,"user_data":null,"user_data_base64":null}},{"address":"aws_launch_configuration.my_web_config","mode":"managed","type":"aws_launch_configuration","name":"my_web_config","provider_name":"aws","schema_version":0,"values":{"associate_public_ip_address":false,"enable_monitoring":true,"ephemeral_block_device":[],"iam_instance_profile":null,"image_id":"ami-09b4b74c","instance_type":"t2.micro","name":"my_web_config","name_prefix":null,"placement_tenancy":null,"security_groups":null,"spot_price":null,"user_data":null,"user_data_base64":null,"vpc_classic_link_id":null,"vpc_classic_link_security_groups":null}}]}},"resource_changes":[{"address":"aws_autoscaling_group.my_asg","mode":"managed","type":"aws_autoscaling_group","name":"my_asg","provider_name":"aws","change":{"actions":["create"],"before":null,"after":{"availability_zones":["us-west-1a"],"desired_capacity":4,"enabled_metrics":null,"force_delete":true,"health_check_grace_period":300,"health_check_type":"ELB","initial_lifecycle_hook":[],"launch_configuration":"my_web_config","launch_template":[],"max_size":5,"metrics_granularity":"1Minute","min_elb_capacity":null,"min_size":1,"mixed_instances_policy":[],"name":"my_asg","name_prefix":null,"placement_group":null,"protect_from_scale_in":false,"suspended_processes":null,"tag":[],"tags":null,"termination_policies":null,"timeouts":null,"wait_for_capacity_timeout":"10m","wait_for_elb_capacity":null},"after_unknown":{"arn":true,"availability_zones":[false],"default_cooldown":true,"id":true,"initial_lifecycle_hook":[],"launch_template":[],"load_balancers":true,"mixed_instances_policy":[],"service_linked_role_arn":true,"tag":[],"target_group_arns":true,"vpc_zone_identifier":true}}},{"address":"aws_autoscaling_group.my_asg2","mode":"managed","type":"aws_autoscaling_group","name":"my_asg2","provider_name":"aws","change":{"actions":["create"],"before":null,"after":{"availability_zones":["us-west-2a"],"desired_capacity":4,"enabled_metrics":null,"force_delete":true,"health_check_grace_period":300,"health_check_type":"ELB","initial_lifecycle_hook":[],"launch_configuration":"my_web_config","launch_template":[],"max_size":6,"metrics_granularity":"1Minute","min_elb_capacity":null,"min_size":1,"mixed_instances_policy":[],"name":"my_asg2","name_prefix":null,"placement_group":null,"protect_from_scale_in":false,"suspended_processes":null,"tag":[],"tags":null,"termination_policies":null,"timeouts":null,"wait_for_capacity_timeout":"10m","wait_for_elb_capacity":null},"after_unknown":{"arn":true,"availability_zones":[false],"default_cooldown":true,"id":true,"initial_lifecycle_hook":[],"launch_template":[],"load_balancers":true,"mixed_instances_policy":[],"service_linked_role_arn":true,"tag":[],"target_group_arns":true,"vpc_zone_identifier":true}}},{"address":"aws_autoscaling_group.my_asg3","mode":"managed","type":"aws_autoscaling_group","name":"my_asg3","provider_name":"aws","change":{"actions":["create"],"before":null,"after":{"availability_zones":["us-west-2b"],"desired_capacity":4,"enabled_metrics":null,"force_delete":true,"health_check_grace_period":300,"health_check_type":"ELB","initial_lifecycle_hook":[],"launch_configuration":"my_web_config","launch_template":[],"max_size":7,"metrics_granularity":"1Minute","min_elb_capacity":null,"min_size":1,"mixed_instances_policy":[],"name":"my_asg3","name_prefix":null,"placement_group":null,"protect_from_scale_in":false,"suspended_processes":null,"tag":[],"tags":null,"termination_policies":null,"timeouts":null,"wait_for_capacity_timeout":"10m","wait_for_elb_capacity":null},"after_unknown":{"arn":true,"availability_zones":[false],"default_cooldown":true,"id":true,"initial_lifecycle_hook":[],"launch_template":[],"load_balancers":true,"mixed_instances_policy":[],"service_linked_role_arn":true,"tag":[],"target_group_arns":true,"vpc_zone_identifier":true}}},{"address":"aws_instance.web","mode":"managed","type":"aws_instance","name":"web","provider_name":"aws","change":{"actions":["create"],"before":null,"after":{"ami":"ami-09b4b74c","credit_specification":[],"disable_api_termination":null,"ebs_optimized":null,"get_password_data":false,"iam_instance_profile":null,"instance_initiated_shutdown_behavior":null,"instance_type":"t2.micro","monitoring":null,"source_dest_check":true,"tags":null,"timeouts":null,"user_data":null,"user_data_base64":null},"after_unknown":{"arn":true,"associate_public_ip_address":true,"availability_zone":true,"cpu_core_count":true,"cpu_threads_per_core":true,"credit_specification":[],"ebs_block_device":true,"ephemeral_block_device":true,"host_id":true,"id":true,"instance_state":true,"ipv6_address_count":true,"ipv6_addresses":true,"key_name":true,"network_interface":true,"network_interface_id":true,"password_data":true,"placement_group":true,"primary_network_interface_id":true,"private_dns":true,"private_ip":true,"public_dns":true,"public_ip":true,"root_block_device":true,"security_groups":true,"subnet_id":true,"tenancy":true,"volume_tags":true,"vpc_security_group_ids":true}}},{"address":"aws_launch_configuration.my_web_config","mode":"managed","type":"aws_launch_configuration","name":"my_web_config","provider_name":"aws","change":{"actions":["create"],"before":null,"after":{"associate_public_ip_address":false,"enable_monitoring":true,"ephemeral_block_device":[],"iam_instance_profile":null,"image_id":"ami-09b4b74c","instance_type":"t2.micro","name":"my_web_config","name_prefix":null,"placement_tenancy":null,"security_groups":null,"spot_price":null,"user_data":null,"user_data_base64":null,"vpc_classic_link_id":null,"vpc_classic_link_security_groups":null},"after_unknown":{"ebs_block_device":true,"ebs_optimized":true,"ephemeral_block_device":[],"id":true,"key_name":true,"root_block_device":true}}}],"configuration":{"provider_config":{"aws":{"name":"aws","expressions":{"region":{"constant_value":"us-west-1"}}}},"root_module":{"resources":[{"address":"aws_autoscaling_group.my_asg","mode":"managed","type":"aws_autoscaling_group","name":"my_asg","provider_config_key":"aws","expressions":{"availability_zones":{"constant_value":["us-west-1a"]},"desired_capacity":{"constant_value":4},"force_delete":{"constant_value":true},"health_check_grace_period":{"constant_value":300},"health_check_type":{"constant_value":"ELB"},"launch_configuration":{"constant_value":"my_web_config"},"max_size":{"constant_value":5},"min_size":{"constant_value":1},"name":{"constant_value":"my_asg"}},"schema_version":0},{"address":"aws_autoscaling_group.my_asg2","mode":"managed","type":"aws_autoscaling_group","name":"my_asg2","provider_config_key":"aws","expressions":{"availability_zones":{"constant_value":["us-west-2a"]},"desired_capacity":{"constant_value":4},"force_delete":{"constant_value":true},"health_check_grace_period":{"constant_value":300},"health_check_type":{"constant_value":"ELB"},"launch_configuration":{"constant_value":"my_web_config"},"max_size":{"constant_value":6},"min_size":{"constant_value":1},"name":{"constant_value":"my_asg2"}},"schema_version":0},{"address":"aws_autoscaling_group.my_asg3","mode":"managed","type":"aws_autoscaling_group","name":"my_asg3","provider_config_key":"aws","expressions":{"availability_zones":{"constant_value":["us-west-2b"]},"desired_capacity":{"constant_value":4},"force_delete":{"constant_value":true},"health_check_grace_period":{"constant_value":300},"health_check_type":{"constant_value":"ELB"},"launch_configuration":{"constant_value":"my_web_config"},"max_size":{"constant_value":7},"min_size":{"constant_value":1},"name":{"constant_value":"my_asg3"}},"schema_version":0},{"address":"aws_instance.web","mode":"managed","type":"aws_instance","name":"web","provider_config_key":"aws","expressions":{"ami":{"constant_value":"ami-09b4b74c"},"instance_type":{"constant_value":"t2.micro"}},"schema_version":1},{"address":"aws_launch_configuration.my_web_config","mode":"managed","type":"aws_launch_configuration","name":"my_web_config","provider_config_key":"aws","expressions":{"image_id":{"constant_value":"ami-09b4b74c"},"instance_type":{"constant_value":"t2.micro"},"name":{"constant_value":"my_web_config"}},"schema_version":0}]}}}

Additional Info

[tsandall]

The policy in the tutorial assumes the input is generated by another tool (tfjson). The fix for this will be to update the tutorial to support the plan format that Terraform v0.12 supports.

[ksexton]

Just a note that newer versions of terraform can output json natively.

terraform show -json tfplan_large.binary > tfplan_large.json

[midnightrun]

tfjson seems to be incompatible with the new generated format of 0.12.

Steps to reproduce:

• Create main.tf

provider "aws" {
  region = "us-east-1"
}

resource "aws_vpc" "main" {
  cidr_block = "10.0.0.0/16"
}

• Generate plan with terraform plan --out tfplan.binary
• Convert Terraform output to Json via tfson tfplan.binary > tfplan.json

Expected behaviour
A generated Json file.

Actual behaviour
Getting the error message that not a valid plan file was given.

[ferhaty]

Faced the same issue and using terraform show -json now...

[midnightrun]

@ferhaty problem with that, is the output can't to be proceeded by OPA. Following the tutorial I get following score:

opa eval --data terraform.rego --input tfplan.json "data.terraform.analysis.score"
{
  "result": [
    {
      "expressions": [
        {
          "value": 0,
          "text": "data.terraform.analysis.score",
          "location": {
            "row": 1,
            "col": 1
          }
        }
      ]
    }
  ]
}

Could you verify that that is working for you?

[tsandall]

@midnightrun see my comment above. We're planning to update the tutorial in the week or two.

[mbowen13]

Just a note that newer versions of terraform can output json natively.

terraform show -json tfplan_large.binary > tfplan_large.json

This only works with v0.12 or later.

@tsandall
As is, I cannot complete the tutorial with TF v0.11 because of an open issue in tfjson:
#7

[rahulkiit]

Hi,

I am using terraform json feature to generate json output and after tweaking the policy it's working.

terraform show -json tfplan.output | jq ".resource_changes | map ({(.type|tostring) : .}) | add" > tfplan.json tfplan.json

https://github.com/rahulkiit/opaterraform

[tsandall]

@vgramer is planning to work on this in the next week or two.