Helm-operator: Phase of a HelmRelease with git chart source sets release as ChartFetchFailed on startup

[nweedo3]

Describe the bug

When the helm operator starts, it sets the phase of all HelmReleases which have a git chart source to ChartFetchFailed, due to either the git mirror not being set up yet or the repository has not yet been cloned. For example:

ts=2020-05-21T11:22:48.372550544Z caller=release.go:81 component=release release=masked targetNamespace=default resource=default:helmrelease/masked helmVersion=v2 error="failed to prepare chart for release: chart not ready: no existing git mirror found"
ts=2020-05-21T11:22:48.373868203Z caller=release.go:81 component=release release=masked2 targetNamespace=default resource=default:helmrelease/masked2 helmVersion=v2 error="failed to prepare chart for release: chart not ready: git repo has not been cloned yet"

The HelmRelease will remain in this phase until a change has been pushed which affects that release.

To Reproduce

Steps to reproduce the behaviour:

1. Have a pre-existing HelmRelease which has a git chart source
2. Restart helm-operator

Expected behavior

On startup, either:

• the operator should wait for git chart sources to be set up before attempting chart synchronisation
• the operator should rectify the phase of each HelmRelease once the chart source has been set up successfully

Logs

Relevant logs are above.

Additional context

• Helm Operator version: 1.0.1
• Kubernetes version: 1.16
• Git provider: GitHub
• Helm version: v2.16.6

[hiddeco]

This should have been fixed in 1.1.0.

[nweedo3]

This should have been fixed in 1.1.0.

Thanks @hiddeco - I'll check it out when I get the opportunity (hopefully today) and report back.

[nweedo3]

Confirmed fixed in 1.1.0. Thanks @hiddeco!

[sa-spag]

I'm not sure this is fixed. I just rolled out the Helm Operator 1.1.0 and I still notice all HelmReleases leveraging Git repositories report condition ChartFetchFailed and Succeeded:

Status:
  Conditions:
    Last Transition Time:  2020-05-29T02:04:42Z
    Last Update Time:      2020-05-29T02:34:43Z
    Message:               Chart fetch failed for Helm release 'r' in 'n'.
    Reason:                ChartFetchFailed
    Status:                False
    Type:                  ChartFetched
    Last Transition Time:  2020-04-06T07:54:20Z
    Last Update Time:      2020-05-29T11:07:28Z
    Message:               Release was successful for Helm release 'r' in 'n'.
    Reason:                Succeeded
    Status:                True
    Type:                  Released

It is also pretty visible with the following Prometheus query: count(flux_helm_operator_release_condition_info{condition="ChartFetched"} == -1).

[andrew-pickin-epi]

Also seeing the same behaviour. All releases with status true for Release & ChartFetched. After restarting v 1.1.0, all releases have ChartFetch ina failed state and all Releases are good.

We also have promethues alerts so these fire on a restart.