Terraform: "terraform plan" should consider refresh updates as needing to be applied

[apparentlymart]

With the addition of data sources in 0.7, it became possible that a terraform plan -out=tfplan could discover that data source changes are present -- possibly then causing _output_ changes -- even if no resources need to be changed as a result.

Currently there is a hole in the workflow where it's tricky to apply such updates.

Along with the existing logic in the plan command that checks if the diff is non-empty, we should also check to see if the state changed during the refresh step. If any changes are present, we should produce a special sort of plan that includes no changes but includes the updated state:

$ terraform plan -out=tfplan
(usual output from the refresh step elided)

Infrastructure is up-to-date, but changes were detected to data sources or resources
during refresh.

These changes have not yet been recorded in the persistent state. To synchronize the
state with the changes to real-world resources, and update any output whose value
depends on these changes, run the "apply" command:
    terraform apply tfplan

This plan will make no changes to real resources.

It's interesting to note here that it feels a little weird to not spell out exactly which changes this will make, but yet it's completely consistent with the amount of information we give about updates when there _are_ resource changes to apply. Along with this, we may wish to consider including changes to _outputs_ in the diff in both cases, so that people with configurations that are consumed downstream by terraform_remote_state can feel more confident about the impact of their changes on those downstream consumers.

Changes to outputs:
      aws_vpc_id:    "vpc-12345" => "vpc-54321"
      aws_subnet_id: "subnet-12345" => <computed>

[apparentlymart]

Here's a more recent UI mock which incorporates this idea, along with some other potential changes including #17034:

The main interesting things shown in this mock are:

• Explicit warning if any changes are detected when Terraform refreshes existing resources. This serves as context for understanding the plan below it, and draws attention to changes that may not have been intended.
• Data resources are read during the plan step, per #17034.
• Terraform includes in the plan output any data resource whose values have changed since the last apply, as additional context to understand the origin of other changes.
• Changes to output values are shown as part of the plan.
• This also includes the new structured diff output format previously discussed in #15180.

The Terraform team at HashiCorp is currently focused on configuration language improvements and so isn't currently working on the changes shown here, but this mock was created as part of exploring potential future needs during the configuration language improvements design phase.

[ocervell]

Looks great ! Any idea on when this will be released ?

[ktham]

@apparentlymart with TF 0.12 and hcl2 now out, is this issue back on your guys' radar?

The issue I'm running into is that adding additional output variables will not get placed in Terraform state because Terraform Enterprise will report that there are "no changes to be made" after executing a terraform plan.

This is problematic when I want to add output variables in one TFE workspace to be consumed in a different TFE workspace, but without changing any existing resources in the former.

Terraform Enterprise does not give an option to execute a terraform refresh so what needs to happen is for terraform plan to actually update Terraform State during this planning phase even if no actual resources are being changed.

[qrilka]

@ktham did you find any workaround for this?

[ktham]

@qrilka The only workaround I can think of is to force terraform to create a new resource

resource "null_resource" "no_op" {
  // TODO: remove me
}

[qrilka]

In the end we had some real changes in the state. Thanks for this hint @ktham

[red8888]

bump. I rely heavily on outputs and imports but now im thinking I made the wrong decision because of this issue.

Its scary to not see what the changed outputs will be before running apply when you know other workspaces depend on those output values.

I eventually wanted to switch to enterprise so I could use run triggers but not before I can see output changes as a diff in plan output