Terraform: lookup fails on a JSON input

[ofer-velich]

Hi there,

Thank you for opening an issue. Please note that we try to keep the Terraform issue tracker reserved for bug reports and feature requests. For general usage questions, please see: https://www.terraform.io/community.html.

If your issue relates to a specific Terraform provider, please open it in the provider's own repository. The index of providers is at https://github.com/terraform-providers .

Terraform Version

Terraform v0.10.6

Terraform Configuration Files

module A

data "http" "json_data" {
  url = "http://myservice/jsondata"

  # Optional request headers
  request_headers {
    "Accept" = "application/json"
  }
}

output "json_data_key" {
  value = "${lookup(data.http.json_data.body, "mykey")}"
}
````
main.tf
----------------
```hcl
provider "aws" {
  region = "${var.region}"
  version = "~> 0.1"
}

module "moduleA" {
  source = "../../../terraform-modules/moduleA"
}

resource "aws_instance" "example" {
  ami = "ami-2757f631"
  instance_type = "${module.moduleA.json_data_key}"
}
````

### Debug Output

Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

data.http.json_data: Refreshing state...

---

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:

• create

Terraform will perform the following actions:

• aws_instance.example
  id:
  ami: "ami-2757f631"
  associate_public_ip_address:
  availability_zone:
  ebs_block_device.#:
  ephemeral_block_device.#:
  instance_state:
  instance_type: "${module.moduleA.json_data_key}"
  ipv6_address_count:
  ipv6_addresses.#:
  key_name:
  network_interface.#:
  network_interface_id:
  placement_group:
  primary_network_interface_id:
  private_dns:
  private_ip:
  public_dns:
  public_ip:
  root_block_device.#:
  security_groups.#:
  source_dest_check: "true"
  subnet_id:
  tenancy:
  volume_tags.%:
  vpc_security_group_ids.#:

### Expected Behavior
Lookup should not fails ... 
The `http://myservice/jsondata` returns a valid json *string*
If i look on the plan output with out running the lookup on the returned json data i can see:

instance_type: "{\"instance_type\":\"c4.large\"}"
```

I would expect that the json would get decoded to a map object and i will be able to run lookup function on it

[ozbillwang]

Use the official example (because its url is alive).

data "http" "example" {
  url = "https://checkpoint-api.hashicorp.com/v1/check/terraform"

  # Optional request headers
  request_headers {
    "Accept" = "application/json"
  }
}

# Sample data:
 {\"product\":\"terraform\",\"current_version\":\"0.10.6\",\"current_release\":1505830478,\"current_download_url\":\"https://terraform.io/downloads.html\",\"current_changelog_url\":\"https://github.com/hashicorp/terraform/blob/v0.10.6/CHANGELOG.md\",\"project_website\":\"https://www.terraform.io\",\"alerts\":[]}
output "product" {
  value = "${lookup(data.http.example.body, "product")}"
}

When run with debug option, I see the error. (But it doesn't display in normal mode)

2017/09/23 03:16:50 [DEBUG] ReferenceTransformer: "data.http.example" references: []
2017/09/23 03:16:50 [WARN] Output interpolation "product" failed: At column 3, line 1: lookup: argument 1 should be type map, got type string in:
${lookup(data.http.example.body, "product")}

So the body output is a string only. That's why it doesn't work with your codes. Because lookup function is only used for map

You need to convert the string to map first.

[ozbillwang]

Seems you need to write a new built-in Functions json2map and add in below file.

terraform/config/interpolate_funcs.go

Refer codes :

https://www.socketloop.com/tutorials/golang-how-to-convert-json-string-to-map-and-slice

https://stackoverflow.com/questions/11066946/partly-json-unmarshal-into-a-map-in-go

[apparentlymart]

Hi @ofer-velich!

As @ozbillwang said, the problem here is that the http data source just returns the raw response body string, without decoding it.

At present Terraform does not have a JSON decoder, because the type system for the configuration language can't represent the interface for such a function (it's a function that takes a string and returns _any type_ based on the input).

The good news is that we're currently working on improving the configuration language to support such things, and once done planning to introduce a new function jsondecode that would allow something like the following to work:

# NOT YET IMPLEMENTED and may change before release

data "http" "json_data" {
  url = "http://example.com/jsondata"

  # Optional request headers
  request_headers {
    "Accept" = "application/json"
  }
}

output "json_data_key" {
  value = "${jsondecode(data.http.json_data.body)["mykey"]}"
}

This work is underway, but it's a big project so we're going to be releasing these new features gradually over a few separate releases to reduce risk. We'll have more to share on this soon.

[apparentlymart]

Hi all,

I was going to verify a variant of the original configuration here in Terraform v0.12.0-alpha1, but sadly a build of the http provider was not included in that release, so I had to settle for a placeholder string instead:

locals {
  placeholder_json = <<EOT
{"country_iso":"US"}
EOT
}

output "detected_country" {
  value = jsondecode(local.placeholder_json)["country_iso"]
}

$ terraform apply

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

detected_country = US

While I used a local value string here instead of a data source, this jsondecode function would work equally well for data.http.json_data.body as I showed in my previous example.

There's some more discussion about this new jsondecode function over in #10363, which this became essentially a duplicate of.

Since this feature is already merged in master and verified in the alpha release, I'm going to close this. This feature will also be included in the forthcoming v0.12.0 final release. Thanks for sharing this use-case, and thanks for your patience while we laid the groundwork to make this possible.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.