Terraform-provider-vault: Feature request: Ability to query if a secret path exists and to initialise a path only if not already present.

[queglay]

This is a feature request. It would help to have the ability to initialise paths and values only if data is not already present there.

I don't believe it's possible either to query if a path exists with the data vault_generic_secret resource to solve the problem conditionally. If the path doesn't exist, the data vault_generic_secret resource will fail. Perhaps it could return another attribute "present" to allows us to know.

This is problematic because if we want terraform to initialise a new path and contents within it only if the data is not present, I don't think we can do this.
The only option we have is for terraform to always create a path and overwrite existing data which is no good on subsequent terraform applies.

Terraform Version

0.13.5

[catsby]

Hey @queglay - This is a tough one, I'm not sure if we can accommodate it. A basic feature of Vault is that it generally doesn't distinguish between something being "not found" and the authenticated user lacking sufficient privileges. While you could query to see if that path is available you can't know for sure if it's available, or if you lack privileges to read the contents. Does that make sense?

[queglay]

Thanks for your reply @catsby! Since it would be for initialisation, privileges should be provided to the initialisation process.
It is possible via the command line for a similar equivalent, like the -cas=0 constraint (https://www.vaultproject.io/docs/secrets/kv/kv-v2.html). This can ensure a value is always present, and it also helps if you want to constrain users to update values and not create new ones, dirtying up path structure.

I got pretty close with this attempt below, but it revealed another bug to me.

# This module will initlise vault values to a default if not present or if already matching an existing default
# This allows us to know if a user has configured to a non default value, and if so, leave it in place.

locals {
  path = "${var.mount_path}/${var.resourcetier}/${var.secret_name}"
}

resource "null_resource" "init_secret" { # init a secret if empty
  provisioner "local-exec" {
    interpreter = ["/bin/bash", "-c"]
    command = <<EOT
      vault kv put -cas=0 "${local.path}" value="" || echo "Value is already initialised / non-zero exit code"
EOT
  }
}

data "vault_generic_secret" "vault_map" { # Get the map of data at the path
  depends_on = [null_resource.init_secret]

  count = var.restore_defaults || var.init ? 0 : 1
  path = local.path
}

locals {
  system_default = var.system_default # The system default map will define the value if value is not present, or value matches a preexisting default.
   # If a present value is different to a present default, retain the vault value.  Else use the system default.
   # We could use the kv put -patch option with a write, but this could increment versions unnecersarily.
  vault_map = element( concat( data.vault_generic_secret.vault_map.*.data, list({}) ), 0 )
  secret_value = contains( keys(local.vault_map), "value" ) && contains( keys(local.vault_map), "default" ) && lookup( local.vault_map, "value", "" ) != lookup( local.vault_map, "default", "") ? lookup( local.vault_map, "value", "") : local.system_default["default"] 
  secret_map = var.restore_defaults ? tomap( {"value" = local.system_default["default"] } ) : tomap( {"value" = local.secret_value } )
  result_map = merge( local.system_default, local.secret_map )
}

resource "vault_generic_secret" "vault_map_output" {
  count = var.init ? 0 : 1
  path = local.path
  data_json = jsonencode( local.result_map )
}

Although data "vault_generic_secret" "vault_map" uses depends_on = [null_resource.init_secret], it will actually error if the path doesn't exist on the first run during plan. If it respected the dependency, it would have some idea that the path at least can't be known until the first dependent operation occurred. To work around that I have to do one run with the cas=0 init line first. Then subsequent runs can use the rest of the set operations.

It's possible two things could work better:

• vault_generic_secret resource could allow use of cas=0 arg
• depends on should be able to withhold evaluation of the vars in the block until the upstream dependency is resolved.

Sidenote: In case anyone else finds the example useful the functionality also updates a values field if it matched a present default field with some hairy logic. If a "value" field matches an existing "default" field (or value is unset), the default can be updated by some schema, and the value will also automatically update. Only if the user has set the value to some non default value will it be preserved. Automation updates present values to some new default only if the user has not already customised the value.

[bcsgh]

I too would like to see something like the cas=0 option for this resource type. That said, I'd prefer it to take a more self explanatory form like create_if_not_present=true, ignore_value=true or update_value=false.

I'd actually be fine with a solution that results in a resource that can't be created; there are likely secrets that, by design, terraform would never be able to populate with valid data. In those cases, forcing the user to take external action ("Go talk to Mordac in IT for the correct value") could well be the only valid solution. This would still leave open a few options:

• Applying the resource without the flag in question, populating it correctly, restore the flag and apply again.
• Manual create the resource and then terraform import it.

Neither is particularly ideal but, in some cases, constraints outside terraform may render anything else impracticable.

[williamclot]

I agree with the above, having such a feature would be useful!