Terraform-provider-helm: Support for set_file?

[sebbsan]

Terraform Version

Terraform v0.11.13
+ provider.azurerm v1.22.1
+ provider.helm v0.9.0
+ provider.kubernetes v1.5.2
+ provider.null v2.0.0
+ provider.random v2.0.0
+ provider.template v2.1.0

Affected Resource(s)

• helm_resource

Terraform Configuration Files

This is an example with set_string which does not work, because it seems like the output out of my terraform template files - for example logstash-input.tpl is processed again through the Helm template language, and also I have to escape period and other things in my configuration values, which is eather unintuitive and makes it harder to re-use values that are falling out of other Terraform providers to serve as input for a Helm chart

# install logstash

data "template_file" "logstash-input" {
  template = "${file("logstash-input.tpl")}"

  vars = {
    kafka_brokers    = "${var.kafka_brokers}"
    consumer_threads = "6"
  }
}

data "template_file" "logstash-filter" {
  template = "${file("logstash-filter.tpl")}"
}

data "template_file" "logstash-output" {
  template = "${file("logstash-output.tpl")}"

  vars = {
    elastic_search_host     = "${var.elastic_endpoint}"
    elastic_search_user     = "${var.elastic_user}"
    elastic_search_password = "${var.elastic_password}"
  }
}

resource "helm_release" "logstash-reporting" {
  name      = "logstash-reporting"
  chart     = "stable/logstash"
  namespace = "default"

  set_string {
    name  = "inputs.main"
    value = "${data.template_file.logstash-input.rendered}"
  }

  set_string {
    name  = "filters.main"
    value = "${data.template_file.logstash-filter.rendered}"
  }

  set_string {
    name  = "outputs.main"
    value = "${data.template_file.logstash-output.rendered}"
  }

  set {
    name  = "persistence.enabled"
    value = "false"
  }

  set {
    name  = "persistence.enabled"
    value = "false"
  }

  set {
    name  = "persistence.enabled"
    value = "false"
  }

  set_string {
    name  = "config.logstashinputkafkabrokers"
    value = "${var.kafka_brokers}"
  }

  set_string {
    name  = "config.logstashinputconsumerthreads"
    value = "6"
  }

  set_string {
    name  = "config.logstashoutputeshost"
    value = "${var.elastic_endpoint}"
  }

  set_string {
    name  = "config.logstashoutputesuser"
    value = "${var.elastic_user}"
  }

  set_string {
    name  = "config.logstashoutputespassword"
    value = "${var.elastic_password}"
  }

  depends_on = ["null_resource.helm_init"]
}

variable "elastic_endpoint" {
  default = "https://4099eea1dee04a478a6e990e3ea1d7b2.eu-central-1.aws.cloud.es.io:9243"
}

variable "elastic_user" {
  default = "ssai"
}

variable "elastic_password" {}

Debug Output

When executing terraform plan, Terraform creates a proper value out of my configuration above, for example:

      set_string.2838357629.value: "filter {\n  json { source => \"message\" }\n  geoip {\n    source => \"[user][rparam][ip]\"\n  }\n  useragent {\n    source => \"[user][rparam][ua]\"\n  }\n  date {\n    match => [ \"[@metadata][kafka][timestamp]\", \"UNIX_MS\" ]\n  }\n  mutate { add_field => { \"[@metadata][id]\" => \"%{[id]}\" } }\n  mutate { add_field => { \"[@metadata][eid]\" => \"%{[eid]}\" } }\n  mutate { remove_field => [ \"message\", \"id\", \"eid\", \"@version\", \"tags\" ] }\n  if \"[user]\" {\n    mutate { remove_field => [ \"[user][rparam][ip]\" ] }\n  }\n}\n"

But when applying the plan, it runs through the Helm templating language (presumably) and it gives me an error:

Error: Error applying plan:

1 error(s) occurred:

* helm_release.logstash-reporting: 1 error(s) occurred:

* helm_release.logstash-reporting: failed parsing key "filters.main" with value filter {
  json { source => "message" }
  geoip {
    source => "[user][rparam][ip]"
  }
  useragent {
    source => "[user][rparam][ua]"
  }
  date {
    match => [ "[@metadata][kafka][timestamp]", "UNIX_MS" ]
  }
  mutate { add_field => { "[@metadata][id]" => "%{[id]}" } }
  mutate { add_field => { "[@metadata][eid]" => "%{[eid]}" } }
  mutate { remove_field => [ "message", "id", "eid", "@version", "tags" ] }
  if "[user]" {
    mutate { remove_field => [ "[user][rparam][ip]" ] }
  }
}
, key " \"id\"" has no value (cannot end with ,)

Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure.

Expected Behavior

Support of Helm --set-file option to set large values, which may have been generated by Terraform template language, or other means

Actual Behavior

trying to work it around as specified above, but doesn't work

Steps to Reproduce

1. terraform apply

Important Factoids

References

[legal90]

You can try using values instead:

resource "helm_release" "logstash-reporting" {
  name      = "logstash-reporting"
  chart     = "stable/logstash"
  namespace = "default"

  values ["${data.template_file.logstash-input.rendered}"]

  #...<other options if needed>
}

P.s. The content of your template logstash-input.tpl should have a nesting in inputs.main level, e.q.:

inputs:
  main: |-
    #  Here is
    #  your file
    #  content

[sebbsan]

this works - thank you very much, Sir!