Terraform-provider-helm: Show the diff manifest in plan.

[mcanevet]

Unless I'm wrong, this provider will not detect changes if neither the chart version, nor the values changed. However, one can have made manual changes using the k8s api (update or delete objects), and this provider should detect this and reconcile using what is described in the Terraform manifest.

I think we can achieve this using the helm diff plugin, we could store the current state in a computed field and then, find a way to compare it to what it will deploy.

[mcanevet]

I tried to implement this last week end but it is not that easy. After discussing about that on #committer-terraform on HashCorp's slack, I had some inputs from @apparentlymart. What we need is:

• a manifest computed argument. This part is easy, I'll probably prepare a PR for that.
• Launch a dry-run of the release to refresh the manifest argument in the CustomizeDIff function. But this may have some side effects.

[mcanevet]

I copy/paste here the discussion I just had on Slack regarding this issue:

Hello,
I need some advise regarding a contribution I'd like to make to the helm_release resource of the helm provider. The "version" argument is "optional", "computed". If one does not specify it, the latest version of the helm chart is taken on creation. When a new version is released, I would expect that terraform tries to upgrade the release to the latest version, but it is not the case because Terraform does not know that a new version is available.
I know how to fetch the latest version by doing a dry-un upgrade of a helm release, but what I don't really know is in what part of the code I should populate the new value of the "version" argument so that terraform can propose an upgrade.

rileykarson [5:32 PM]
We use datasources in the Google provider so that Terraform knows an upgrade is available in similar cases, if it helps: https://www.terraform.io/docs/providers/google/d/google_container_engine_versions.html (edited)
Terraform by HashiCorp
Google: google_container_engine_versions - Terraform by HashiCorp
Provides lists of available Google Container Engine versions for masters and nodes.

mcanevet [5:39 PM]
hmmm ok. That would do the trick for this use case. But I have another contribution in mind that would consist of adding a "manifest" computed argument to store the generated manifest. This would allow to show a clean difference of what would be really deployed in a k8s cluster with the new diff renderer of Terraform v0.12 (https://github.com/hashicorp/terraform/issues/15180#issuecomment-435241641). But for this use case I need to populate the "manifest" argument with a dry-run

apparentlymart [5:40 PM]
another possibility, if the resource code is able to detect that a change is needed automatically, would be to implement the resource-level `CustomizeDiff` function to fill in a new value for that computed attribute where appropriate. With that said, I think right now there might be a missing piece for that where the SDK API doesn't have a mechanism to distinguish between "set in configuration" vs. "computed previously" in `CustomizeDiff`. :thinking_face:
That potential problem aside, one of the roles of `CustomizeDiff` is to notice when one change will cause a knock-on side-effect on other attributes, so perhaps it's a reasonable fit here
I was trying some similar things for HashiCorp Nomad over here: https://github.com/terraform-providers/terraform-provider-nomad/pull/15 though I've not had time to follow through on that and finish it up

bflad [5:44 PM]
`ResourceDiff.SetNew()` and `ResourceDiff.SetNewComputed()` are how attributes can be marked with side-effect changes within `CustomizeDiff` if I remember correctly: https://godoc.org/github.com/hashicorp/terraform/helper/schema#ResourceDiff.SetNew (edited)

apparentlymart [5:45 PM]
:nod-neutral:  right
there are some higher-level helpers in this package for `CustomizeDiff` but I don't think any of them are quite right for this case because it sounds like the mapping from the inputs to the proposed "manifest" will be quite complex in its own right https://godoc.org/github.com/hashicorp/terraform/helper/customdiff
godoc.org
Package customdiff
Package customdiff provides a set of reusable and composable functions to enable more "declarative" use of the CustomizeDiff mechanism available for resources in package helper/schema.

mcanevet [5:51 PM]
@apparentlymart @bflad, ok, there is already a CustomizeDiff function : https://github.com/terraform-providers/terraform-provider-helm/blob/master/helm/resource_release.go#L271-L277, should I launch the dry-run here an do a SetNewComputed("version", ...) and SetNewComputed("manifest", ...)?

apparentlymart [5:56 PM]
Yeah, that'd be the general idea I think. One thing to watch out for (which I think the existing implementation isn't handling, actually) is that during the plan phase some of the settings might be not known yet because they depend on attributes from other resources that have not yet been completed. You can use `d.NewValueKnown` to test if a value is known; if it isn't, you'd need to set these attributes as themselves "computed" so that the unknown-ness can propagate down to other resources and show properly in the rendered diff.
If you try to build a partial thing with some of the inputs unknown then it's likely that you'd get the dreaded "diffs didn't match during apply" error once the results are finally known, since Terraform requires that the final diff created during the apply phase must match the diff that was recorded in the plan except that previously-unknown values can become known
(my nomad prototype has a similar bug, in fact... it would fail if the job description string were unknown during planning)

[ausov]

Would it be possible to (re-)use "helm-diff" Helm plugin (like in "helmfile")?

[osterman]

@ausov see the latest contribution by @mumoshu https://github.com/mumoshu/terraform-provider-helmfile

Best of both worlds. Helm+diff with Helmfile+terraform

[dudicoco]

Is there an update on this very critical issue? Running terraform plan without showing the diff in helm is unusable for production in my opinion.

Currently the helmfile provider is not an option as the diff is not working properly.

[gordonbondon]

I did an initial WIP on trying to use helm-diff here #553, please give it a try.