Helmfile: remove "helm diff" errors when doing apply

[sgandon]

when doing a helmfile apply there are errors coming from the helm diff plugin and this is really confusing.
I understand this is the --detailed-exitcode option of helm diff that does this but that would be nice to not have those errors displayed.
I would suggest either file the output in helmfile or may be contribute in to helm diff repos to have a quiet version.

[mumoshu]

Thanks for the detailed explanation!
Yeah this should definitely be fixed.

--quiet sounded good, but it seems to make it impossible for us to show diff when it exists.

I'd tweak the log level for this helm diff runs probably to DEBUG in case of "successful statuses"(not necessarily 0 in this case), in othercases INFO(default) as it is today.

[savar]

--deleted-- my comment, sorry

[madAndroid]

We're hitting up against this as well .. is it possible to have the --detailed-exit codes be more meaningful? for example, in Terraform, this flag exits 2 when there are changes, instead of 1 - which means that it's easier to determine when there actually _are_ changes, or if the diff command has actually failed "properly"

[mumoshu]

@madAndroid Thanks fo reporting!

That's what --detailed-exitcode should do in the first place! So I took some time to investigate what's happening, and found a possibly relevant bug. I've fixed it in #586

[mumoshu]

@sgandon I've submitted #586. One of its changes is to reduce confusing outputs like that. Would you mind taking a look?

[madAndroid]

Thanks @mumoshu - looking forward to the next release that contains this fix, it will make our wrapper script behave a lot more sensibly :)

[mumoshu]

@madAndroid Out of curiosity, did your helmfile diff --detailed-exitcode returned 1 when it should return 2, and it happened when you had helmfiles: (sub-helmfiles) in your helmfile.yaml?

If that's the case, #586 should fix that.

--detailed-exitcode should have returned 2 if you had flat helmfile.yaml(no helmfiles: specified) and there were diffs.

[madAndroid]

@mumoshu - Actually, we don't use sub-helmfiles at all - just a single standard Helmfile, using:

helmfile -f helmfiles/dev.yaml diff --suppress-secrets --detailed-exitcode

and this would then return 1 instead of 2 when it picked up changes

[mumoshu]

@madAndroid Thanks! It should have returned 2 when there were change(s). Anyways, I hope the upcoming release with the fix works for you 🤔

[madAndroid]

@mumoshu - thanks! I should be able to try it out sometime this week .. will be sure to keep you posted on the results :)

[madAndroid]

@mumoshu - it's still happening, I'm afraid - should I log a new issue?

Here's the output:

git:master  ✭ ◼ ❯❯❯❯ helmfile -b /usr/local/bin/helm -e mgmt-prime -l name=jenkins -f helmfiles/mgmt-prime.yaml diff --suppress-secrets --detailed-exitcode
Decrypting secret ../helm_vars/helmfile_secure/prime.yaml
Decrypting ../helm_vars/helmfile_secure/prime.yaml

Decrypting secret ../helm_vars/helmfile_secure/jnlp-agent-k8s-config.yaml
Decrypting ../helm_vars/helmfile_secure/jnlp-agent-k8s-config.yaml

Decrypting secret ../helm_vars/helmfile_secure/prime.yaml
Decrypting ../helm_vars/helmfile_secure/prime.yaml

Decrypting secret ../helm_vars/helmfile_secure/jnlp-agent-k8s-config.yaml
Decrypting ../helm_vars/helmfile_secure/jnlp-agent-k8s-config.yaml

Adding repo stable https://kubernetes-charts.storage.googleapis.com
"stable" has been added to your repositories

Adding repo incubator https://kubernetes-charts-incubator.storage.googleapis.com
"incubator" has been added to your repositories

Adding repo cyclone s3://cyclone-helm-repo.dev.develop.farm/charts
"cyclone" has been added to your repositories

Updating repo
Hang tight while we grab the latest from your chart repositories...
...Skip local chart repository
...Successfully got an update from the "incubator" chart repository
...Successfully got an update from the "stable" chart repository
...Successfully got an update from the "cyclone" chart repository
Update Complete. ⎈ Happy Helming!⎈

Comparing jenkins stable/jenkins
in helmfiles/mgmt-prime.yaml: failed processing release jenkins: helm exited with status 1:
  Error: identified at least one change, exiting with non-zero exit code (detailed-exitcode parameter enabled)
  Error: plugin "diff" exited with error

git:master ⏎  ✭ ◼ ❯❯❯❯ echo $?
1

When I run with --log-level debug, I can see a diff, but it still exits with 1 instead of 2

[mumoshu]

@macAndroid Which version of helm are you using? It need to be gteq 2.11 because we need https://github.com/helm/helm/commit/3e5b4066d2a067ef49f6ca39b3f84d2ebcad25ba to make helm return non-zero exit codes other than 1

[madAndroid]

@mumoshu - ahhh! .. that's probably the reason .. we've got:

helm version
Client: &version.Version{SemVer:"v2.9.1", GitCommit:"20adb27c7c5868466912eebdf6664e7390ebe710", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.9.1", GitCommit:"20adb27c7c5868466912eebdf6664e7390ebe710", GitTreeState:"clean"}

I'll see if we can try and bump to a version > 2.11.x soon, and then test again..
thanks for all your help! :)

[madAndroid]

@mumoshu - Okay, even with a newer version of helmfile and helm, even though I'm getting the correct exit code of 2, I'm seeing this in the output:

identified at least one change, exiting with non-zero exit code (detailed-exitcode parameter enabled)

in helmfiles/mgmt-prime.yaml: failed processing release jenkins: helm exited with status 2:
  Error: identified at least one change, exiting with non-zero exit code (detailed-exitcode parameter enabled)
  Error: plugin "diff" exited with error
2

The Error output here is misleading IMO

The versions:

helm version
Client: &version.Version{SemVer:"v2.13.1", GitCommit:"618447cbf203d147601b4b9bd7f8c37a5d39fbb4", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.13.1", GitCommit:"618447cbf203d147601b4b9bd7f8c37a5d39fbb4", GitTreeState:"clean"}

helmfile --version
helmfile version v0.64.2

... should I open a new issue for this?

[mumoshu]

@madAndroid Hey! Assuming you're talking about the U/X - yes, it's confusing.

What should helmfile show you when helm diff detected changes?

To be clear, the original helm diff shows plugin "diff" exited with error when it detected changes. Should we format/strip the confusing messages from helm diff and show something more concise?

[mumoshu]

@madAndroid I'm ok with talking in this thread but you'll get more attention if you've created a dedicated issue for it!

[madAndroid]

@mumoshu - thanks, I've created a new issue, since this is continuously being misinterpreted by our team :) thanks in advance!