Consul: import command failing

Created on 23 Oct 2018 · 3Comments · Source: hashicorp/consul

I have below json
{
"bar":"test"
}

Overview of the Issue

am trying to import above json in consul am getting below error.
consul kv import @abc.json
Cannot unmarshal data: json: cannot unmarshal object into Go value of type []*impexp.Entry

consul version: Consul v1.2.2

version ubuntu 16.4 LTS

Source

pritam97

All 3 comments

The consul kv import command isn't very well documented beyond its direct relationship with the corresponding consul kv export command. The documentation assumes you will not be directly generating the input data, and as such the format is only documented in the code itself.

The format is defined here:

The input json file is expected to be an array of objects, each one with the following format:

{
  "key":   "YOUR_KEY_NAME_STRING",
  "value": "BASE64_ENCODED_VALUE_STRING",
  "flags": optional_flags_integer
}

To achieve what you intended above it would be:

# base64 encode the value payload
$ echo -n "test" | base64
dGVzdA==

# generate the file
$ cat >abc.json <<EOF
[{
  "key":   "bar",
  "value": "dGVzdA=="
}]
EOF

# import the data
$ consul kv import @abc.json
Imported: bar

# verify the results
$ consul kv get -recurse
bar:test
$ consul kv get -detailed bar
CreateIndex      55
Flags            0
Key              bar
LockIndex        0
ModifyIndex      55
Session          -
Value            test

I hope this clarifies the command.

rboyer on 23 Oct 2018

👍1

@rboyer : Thank you for quick response.
I have nested json with more than 1000 keys/values, it is not feasible for me convert each key into base64 format. do you have any idea about any utility that can easily import json. and my json contains the LIST as well.

Thanks you
Pritam K

pritam97 on 23 Oct 2018

👎1

There are many ways to bend your json into the correct shape. The full solution is going to be very specific to your inputs, but here's a way forward for you utilizing the fantastic tool jq:

# Your example input, shaped similarly to your original inquiry.
$ cat > orig.json <<EOF
{
  "key1" : "value1",
  "key2" : "value2"
}
EOF

# Use the jq tool to walk the map of kv pairs and reshape it
# into the format consul kv import expects.
$ <orig.json jq '. | to_entries | . | map({key:.key, value:(.value| @base64)})' > new.json
$ cat new.json
[
  {
    "key": "key1",
    "value": "dmFsdWUx"
  },
  {
    "key": "key2",
    "value": "dmFsdWUy"
  }
]

# verify the results
$ consul kv import @new.json
Imported: key1
Imported: key2
$ consul kv get -recurse
key1:value1
key2:value2

I hope this helps!

As this isn't a bug in Consul I will close this issue.

Issues on GitHub for Consul are intended to be related to bugs or feature requests, so we recommend using our other community resources instead of asking here in the future.