Terraform-provider-helm: rpc error: code = Unavailable desc

[hashibot[bot]]

_This issue was originally opened by @jayaramcloud as hashicorp/terraform#24882. It was migrated here as a result of the provider split. The original body of the issue is below._

---

ambassador.tf file -
resource "kubernetes_namespace" "ambintelb" {
metadata {
annotations = {
name = "ambintelb"
}
labels = {
namespace = "ambintelb"
}
name = "ambintelb"
}
}

data "helm_repository" "ambassador" {
name = "ambassador"
url = "https://www.getambassador.io/"
}

resource "helm_release" "ambassador" {
name = "ambintelb"
chart = "datawire/ambassador"
version = "6.3.2"
namespace = kubernetes_namespace.ambintelb.metadata[0].name
timeout = 3600

set {
name = "enableAES"
value = "false"
}
set {
name = "image.tag"
value = "1.4.2"
}
set {
name = "image.repository"
value = "quay.io/datawire/ambassador"
}
set {
name = "metrics.serviceMonitor.enabled"
value = "true"
}
set {
name = "adminService.create"
value = "true"
}
set {
name = "service.annonations[0]"
value = var.amb_tags["service.annotation"]
}
set {
name = "service.annonations.service.beta.kubernetes.io/aws-load-balancer-internal[0]"
value = var.amb_tags["service.beta.kubernetes.io/aws-load-balancer-internal"]
}

# set {
# name = "service.annotservice.beta.kubernetes.io/aws-load-balancer-internal"
# value = "service.beta.kubernetes.io/aws-load-balancer-internal"
# }
}

variable "amb_tags" {
type = map
default = {
"service.annotation" = "true"
"service.beta.kubernetes.io/aws-load-balancer-internal" = "true"
}
}

---- CRASH DETAILS ---

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
~ update in-place

Terraform will perform the following actions:

# module.common_shared_module.helm_release.ambassador will be updated in-place
~ resource "helm_release" "ambassador" {
atomic = false
chart = "datawire/ambassador"
cleanup_on_fail = false
dependency_update = false
disable_crd_hooks = false
disable_openapi_validation = false
disable_webhooks = false
force_update = false
id = "ambintelb"
max_history = 0
metadata = [
{
chart = "ambassador"
name = "ambintelb"
namespace = "ambintelb"
revision = 2
values = jsonencode(
{
adminService = {
create = true
}
enableAES = false
image = {
repository = "quay.io/datawire/ambassador"
tag = "1.4.2"
}
metrics = {
serviceMonitor = {
enabled = true
}
}
service = {
annonations = [
true,
true,
]
}
}
)
version = "6.3.2"
},
]
name = "ambintelb"
namespace = "ambintelb"
recreate_pods = false
render_subchart_notes = true
replace = false
reset_values = false
reuse_values = false
skip_crds = false
status = "deployed"
timeout = 3600
verify = false
version = "6.3.2"
wait = true

    set {
        name  = "adminService.create"
        value = "true"
    }
    set {
        name  = "enableAES"
        value = "false"
    }
    set {
        name  = "image.repository"
        value = "quay.io/datawire/ambassador"
    }
    set {
        name  = "image.tag"
        value = "1.4.2"
    }
    set {
        name  = "metrics.serviceMonitor.enabled"
        value = "true"
    }
  + set {
      + name  = "service.annonations.service.beta.kubernetes.io/aws-load-balancer-internal[0]"
      + value = "true"
    }
    set {
        name  = "service.annonations[0]"
        value = "true"
    }
  - set {
      - name  = "service.annonations[1]" -> null
      - value = "true" -> null
    }
}

Plan: 0 to add, 1 to change, 0 to destroy.

Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.

Enter a value: yes

module.common_shared_module.helm_release.ambassador: Modifying... [id=ambintelb]

Error: rpc error: code = Unavailable desc = transport is closing

panic: interface conversion: interface {} is map[string]interface {}, not []interface {}
2020-05-06T12:08:35.477-0600 [DEBUG] plugin.terraform-provider-helm_v1.1.1_x4:
2020-05-06T12:08:35.477-0600 [DEBUG] plugin.terraform-provider-helm_v1.1.1_x4: goroutine 45 [running]:
2020-05-06T12:08:35.477-0600 [DEBUG] plugin.terraform-provider-helm_v1.1.1_x4: helm.sh/helm/v3/pkg/strvals.(parser).key(0xc000355d60, 0xc000911080, 0xc000a51280, 0x7)
2020-05-06T12:08:35.477-0600 [DEBUG] plugin.terraform-provider-helm_v1.1.1_x4: /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-helm/vendor/helm.sh/helm/v3/pkg/strvals/parser.go:173 +0xe04
2020-05-06T12:08:35.477-0600 [DEBUG] plugin.terraform-provider-helm_v1.1.1_x4: helm.sh/helm/v3/pkg/strvals.(parser).key(0xc000355d60, 0xc000910360, 0x1a8ce9d, 0x1d89440)
2020-05-06T12:08:35.477-0600 [DEBUG] plugin.terraform-provider-helm_v1.1.1_x4: /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-helm/vendor/helm.sh/helm/v3/pkg/strvals/parser.go:215 +0x1ad
2020-05-06T12:08:35.477-0600 [DEBUG] plugin.terraform-provider-helm_v1.1.1_x4: helm.sh/helm/v3/pkg/strvals.(parser).parse(0xc000355d60, 0xc000910360, 0x0)
2020-05-06T12:08:35.477-0600 [DEBUG] plugin.terraform-provider-helm_v1.1.1_x4: /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-helm/vendor/helm.sh/helm/v3/pkg/strvals/parser.go:133 +0x38
2020-05-06T12:08:35.477-0600 [DEBUG] plugin.terraform-provider-helm_v1.1.1_x4: helm.sh/helm/v3/pkg/strvals.ParseInto(0xc000b2ad00, 0x1b, 0xc000910360, 0x2, 0x2)
2020-05-06T12:08:35.477-0600 [DEBUG] plugin.terraform-provider-helm_v1.1.1_x4: /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-helm/vendor/helm.sh/helm/v3/pkg/strvals/parser.go:70 +0xbe
2020-05-06T12:08:35.477-0600 [DEBUG] plugin.terraform-provider-helm_v1.1.1_x4: github.com/terraform-providers/terraform-provider-helm/helm.getValues(0xc0006ac850, 0x1f6cef9, 0x18, 0x1be93c0)
2020-05-06T12:08:35.477-0600 [DEBUG] plugin.terraform-provider-helm_v1.1.1_x4: /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-helm/helm/resource_release.go:765 +0x33e
2020-05-06T12:08:35.477-0600 [DEBUG] plugin.terraform-provider-helm_v1.1.1_x4: github.com/terraform-providers/terraform-provider-helm/helm.resourceReleaseUpdate(0xc0006ac850, 0x1d56f80, 0xc00023c240, 0x24, 0x338e820)
2020-05-06T12:08:35.477-0600 [DEBUG] plugin.terraform-provider-helm_v1.1.1_x4: /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-helm/helm/resource_release.go:562 +0x69f
2020-05-06T12:08:35.477-0600 [DEBUG] plugin.terraform-provider-helm_v1.1.1_x4: github.com/hashicorp/terraform-plugin-sdk/helper/schema.(Resource).Apply(0xc0005e3b80, 0xc0007fefa0, 0xc000b7dd20, 0x1d56f80, 0xc00023c240, 0xc000b5cc01, 0xc000b93200, 0xc000b5ccb8)
2020-05-06T12:08:35.477-0600 [DEBUG] plugin.terraform-provider-helm_v1.1.1_x4: /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-helm/vendor/github.com/hashicorp/terraform-plugin-sdk/helper/schema/resource.go:311 +0x263
2020-05-06T12:08:35.477-0600 [DEBUG] plugin.terraform-provider-helm_v1.1.1_x4: github.com/hashicorp/terraform-plugin-sdk/helper/schema.(Provider).Apply(0xc0005a2180, 0xc000a519c8, 0xc0007fefa0, 0xc000b7dd20, 0xc000b50aa8, 0xc00067e501, 0x1c6e8a0)
2020-05-06T12:08:35.477-0600 [DEBUG] plugin.terraform-provider-helm_v1.1.1_x4: /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-helm/vendor/github.com/hashicorp/terraform-plugin-sdk/helper/schema/provider.go:294 +0x99
2020-05-06T12:08:35.477-0600 [DEBUG] plugin.terraform-provider-helm_v1.1.1_x4: github.com/hashicorp/terraform-plugin-sdk/internal/helper/plugin.(GRPCProviderServer).ApplyResourceChange(0xc0000c2fe0, 0x2360c60, 0xc0009c64b0, 0xc0000825a0, 0xc0000c2fe0, 0xc0009c64b0, 0xc0009b2b30)
2020-05-06T12:08:35.477-0600 [DEBUG] plugin.terraform-provider-helm_v1.1.1_x4: /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-helm/vendor/github.com/hashicorp/terraform-plugin-sdk/internal/helper/plugin/grpc_provider.go:885 +0x8b4
2020-05-06T12:08:35.477-0600 [DEBUG] plugin.terraform-provider-helm_v1.1.1_x4: github.com/hashicorp/terraform-plugin-sdk/internal/tfplugin5._Provider_ApplyResourceChange_Handler(0x1eb84a0, 0xc0000c2fe0, 0x2360c60, 0xc0009c64b0, 0xc000082540, 0x0, 0x2360c60, 0xc0009c64b0, 0xc000b23b00, 0xcab)
2020-05-06T12:08:35.477-0600 [DEBUG] plugin.terraform-provider-helm_v1.1.1_x4: /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-helm/vendor/github.com/hashicorp/terraform-plugin-sdk/internal/tfplugin5/tfplugin5.pb.go:3189 +0x217
2020-05-06T12:08:35.477-0600 [DEBUG] plugin.terraform-provider-helm_v1.1.1_x4: google.golang.org/grpc.(Server).processUnaryRPC(0xc00008fe00, 0x2387820, 0xc0009ac600, 0xc000142800, 0xc000604d50, 0x33406c0, 0x0, 0x0, 0x0)
2020-05-06T12:08:35.477-0600 [DEBUG] plugin.terraform-provider-helm_v1.1.1_x4: /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-helm/vendor/google.golang.org/grpc/server.go:1024 +0x4f4
2020-05-06T12:08:35.477-0600 [DEBUG] plugin.terraform-provider-helm_v1.1.1_x4: google.golang.org/grpc.(Server).handleStream(0xc00008fe00, 0x2387820, 0xc0009ac600, 0xc000142800, 0x0)
2020-05-06T12:08:35.477-0600 [DEBUG] plugin.terraform-provider-helm_v1.1.1_x4: /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-helm/vendor/google.golang.org/grpc/server.go:1313 +0xd97
2020-05-06T12:08:35.477-0600 [DEBUG] plugin.terraform-provider-helm_v1.1.1_x4: google.golang.org/grpc.(Server).serveStreams.func1.1(0xc0000c0e70, 0xc00008fe00, 0x2387820, 0xc0009ac600, 0xc000142800)
2020-05-06T12:08:35.477-0600 [DEBUG] plugin.terraform-provider-helm_v1.1.1_x4: /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-helm/vendor/google.golang.org/grpc/server.go:722 +0xbb
2020-05-06T12:08:35.477-0600 [DEBUG] plugin.terraform-provider-helm_v1.1.1_x4: created by google.golang.org/grpc.(Server).serveStreams.func1
2020-05-06T12:08:35.477-0600 [DEBUG] plugin.terraform-provider-helm_v1.1.1_x4: /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-helm/vendor/google.golang.org/grpc/server.go:720 +0xa1
2020-05-06T12:08:35.482-0600 [DEBUG] plugin: plugin process exited: path=/home/jay/gameday/superbeast/sift_ops.03May/sandbox-uw2/.terraform/plugins/linux_amd64/terraform-provider-helm_v1.1.1_x4 pid=306369 error="exit status 2"
2020/05/06 12:08:35 [DEBUG] module.common_shared_module.helm_release.ambassador: apply errored, but we're indicating that via the Error pointer rather than returning it: rpc error: code = Unavailable desc = transport is closing
2020/05/06 12:08:35 [TRACE] module.common_shared_module: eval: *terraform.EvalMaybeTainted
2020/05/06 12:08:35 [TRACE] module.common_shared_module: eval: *terraform.EvalWriteState
2020/05/06 12:08:35 [TRACE] EvalWriteState: recording 1 dependencies for module.common_shared_module.helm_release.ambassador
2020/05/06 12:08:35 [TRACE] EvalWriteState: writing current state object for module.common_shared_module.helm_release.ambassador
2020/05/06 12:08:35 [TRACE] module.common_shared_module: eval: *terraform.EvalApplyProvisioners
2020/05/06 12:08:35 [TRACE] EvalApplyProvisioners: helm_release.ambassador is not freshly-created, so no provisioning is required
2020/05/06 12:08:35 [TRACE] module.common_shared_module: eval: *terraform.EvalMaybeTainted
2020/05/06 12:08:35 [TRACE] module.common_shared_module: eval: *terraform.EvalWriteState
2020/05/06 12:08:35 [TRACE] EvalWriteState: recording 1 dependencies for module.common_shared_module.helm_release.ambassador
2020/05/06 12:08:35 [TRACE] EvalWriteState: writing current state object for module.common_shared_module.helm_release.ambassador
2020/05/06 12:08:35 [TRACE] module.common_shared_module: eval: *terraform.EvalIf
2020/05/06 12:08:35 [TRACE] module.common_shared_module: eval: *terraform.EvalIf
2020/05/06 12:08:35 [TRACE] module.common_shared_module: eval: *terraform.EvalWriteDiff
2020/05/06 12:08:35 [TRACE] module.common_shared_module: eval: *terraform.EvalApplyPost
2020/05/06 12:08:35 [ERROR] module.common_shared_module: eval: *terraform.EvalApplyPost, err: rpc error: code = Unavailable desc = transport is closing
2020/05/06 12:08:35 [ERROR] module.common_shared_module: eval: *terraform.EvalSequence, err: rpc error: code = Unavailable desc = transport is closing
2020/05/06 12:08:35 [TRACE] [walkApply] Exiting eval tree: module.common_shared_module.helm_release.ambassador
2020/05/06 12:08:35 [TRACE] vertex "module.common_shared_module.helm_release.ambassador": visit complete
2020/05/06 12:08:35 [TRACE] dag/walk: upstream of "module.common_shared_module.provider.helm (close)" errored, so skipping
2020/05/06 12:08:35 [TRACE] dag/walk: upstream of "meta.count-boundary (EachMode fixup)" errored, so skipping
2020/05/06 12:08:35 [TRACE] dag/walk: upstream of "root" errored, so skipping
2020/05/06 12:08:35 [TRACE] statemgr.Filesystem: creating backup snapshot at terraform.tfstate.backup
2020/05/06 12:08:35 [TRACE] statemgr.Filesystem: state has changed since last snapshot, so incrementing serial to 167
2020/05/06 12:08:35 [TRACE] statemgr.Filesystem: writing snapshot at terraform.tfstate
2020/05/06 12:08:35 [TRACE] statemgr.Filesystem: removing lock metadata file .terraform.tfstate.lock.info
2020/05/06 12:08:35 [TRACE] statemgr.Filesystem: unlocking terraform.tfstate using fcntl flock
2020-05-06T12:08:35.579-0600 [DEBUG] plugin: plugin exited

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

Terraform crashed! This is always indicative of a bug within Terraform.
A crash log has been placed at "crash.log" relative to your current
working directory. It would be immensely helpful if you could please
report the crash with Terraform[1] so that we can fix this.

When reporting bugs, please include your terraform version. That
information is available on the first line of crash.log. You can also
get it by running 'terraform --version' on the command line.

SECURITY WARNING: the "crash.log" file that was created may contain
sensitive information that must be redacted before it is safe to share
on the issue tracker.

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

[zalyoung]

Meet the same issue when remove a helm release by using helm provider.

But when I try the terraform apply multiple times, the issue disappeared

Seems it happens randomly

[ulzeraj]

@zalyoung Setting argument -parallelism=1 works for me although it makes the process a lot slower.

It doesn't fail if I have only one helm_release resource and only after I add a few the error starts to happen.

[apino]

Same problem for me when I try to create more than one resource. The solution of @ulzeraj works for me. Thanks.

[sc250024]

Ditto. This is a bit annoying, and it's a show-stopper for us using this Terraform provider.

[tiparega]

Same problem here, but @ulzeraj solution is not working for me.

[jodem]

Provider 1.30.0 has fixed an issue when using multiple charts in the same stack.
Re-doing terraform init can help to fix.
I also when it's happening like 10 times in a row, double check your chart : I had the error with a SET in the helm_release that was not consistent with the values file. It gave me that error instead of a proper description.

[katlimruiz]

I also had an error with ambassador when using helm_release: this issue, which I was able to unblock.

But then, when I ran helm_release it kept creating for several minutes for several times (Still creating ...).

When I activated HELM_DEBUG and TF_LOG it stuck after creating the CRDs.

When I do it thru command line helm install ... it works normally.