Jq: Cannot select field if field name has dashes

[ilkka]

Given this JSON:

[
    {
        "my-field": "val1",
        "other_field": "valA"
    },
    {
        "my-field": "val2",
        "other_field": "valB"
    }
]

this works:

$ ./jq '[.[]|{other_field}]' < test.json
[
  {
    "other_field": "valA"
  },
  {
    "other_field": "valB"
  }
]

but this does not:

$ ./jq '[.[]|{my-field}]' < test.json
error: syntax error, unexpected '-', expecting '}'
[.[]|{my-field}]
        ^
1 compile error

I think it should since the JSON is valid. Maybe there is a way around this?

[stedolan]

Currently, that gets parsed as a subtraction. You can always explicitly use strings for when your keys don't fit identifier syntax:

{"my-new-field": .["my-old-field"]}

[ilkka]

Guess that makes sense, I just didn't stumble across that syntax :)

[woohgit]

This means I cannot get a value from a json if the key contains a dash:

< json.json jq .stuff.thisisakey-key.identifier

because it'll throws:

error: key is not defined
.stuff.thisisakey-key.identifier
                  ^^^
1 compile error

How can I do it? I tried with "" and so on, but none of them works.

[ghost]

jq '.stuff["thisisakey-key"].identifier' http://stedolan.github.io/jq/manual/#foo it's similar in JavaScript

[zasran]

According to http://stedolan.github.io/jq/manual/ "If the key contains special characters, you need to surround it with double quotes like this: ."foo$"."

However the following does not work:

$ echo '{"a-b":"0"}' | jq '."a-b"'
error: syntax error, unexpected QQSTRING_START, expecting $end
."a-b"
^
1 compile error

This works:

$ echo '{"a-b":"0"}' | jq '.["a-b"]'
"0"

However it's not clear how to make it work if you want to get a nested value:

$ echo '{"a-b":{"c-d":0}}' | jq '.["a-b"]."c-d"'
error: syntax error, unexpected QQSTRING_START, expecting IDENT
.["a-b"]."c-d"
^
1 compile error

$ echo '{"a-b":{"c-d":0}}' | jq '.["a-b"].["c-d"]'
error: syntax error, unexpected '[', expecting IDENT
.["a-b"].["c-d"]
^
1 compile error

Two questions/possible bugs:

Should ."a-b" work (in the example above)?
How to specify c-d (i.e. nested key with special characters)?

[ghost]

Okay, so, to settle this question already:

If the key has no special characters, access it like .this.

If the key has special characters, access it like ["this"].

So, @zasran, what you're looking for is .["a-b"]["c-d"].

[zasran]

Thanks, .["a-b"]["c-d"] does work.

It still looks a bit inconsistent, maybe I don't understand what dot does or something like that (I read the docs at http://stedolan.github.io/jq/manual/).

When there are no special characters this works:

$ echo '{"ab":{"cd":319}}' | jq '.ab.cd'

But if there are special characters it works without the dot (dot actually causes error):

$ echo '{"ab":{"c-d":319}}' | jq '.ab["c-d"]'
319

This seems a bit inconsistent, why is there a dot separator there for non-special character keys and no dot separator when the ["c-d"] format is used? (maybe calling dot separator is not correct but not sure what to call it)

On a similar note, why does the documentation claim that ."foo$" would work (Basic filters section)? As far as I can tell it claims that .foo should be replaced by ."foo$" if the key contains special characters. The .["foo"] version is also mentioned. But what you are saying is that it really should be "foo$".

Example straight from the docs (last .foo example in Basic filters section)):

echo '{"foo": 42}' | jq '."foo"'
error: syntax error, unexpected QQSTRING_START, expecting $end
."foo"
^
1 compile error

Can you please shed some light on this, maybe change the docs so that it's more obvious? Or maybe it's actual bug and it should work as described in the docs?

[nicowilliams]

Dot (.) refers to "the input to the current expression", roughly.

The ."string" syntax (equivalent to .["string"]) is not in a released version, but it is in the master branch.

[nicowilliams]

I'll fix the docs to list what's in 1.3 and what's new since at some point. Also, I think we'll do a new release soon.

[zasran]

@nicowilliams thanks, that clarifies it!

[akram]

Hi all,
@nicowilliams
thank you for the great job, this tool really saves me a lot of time.
Unfortunately I am facing this issue, and it seems that a release was planned for soon.
correct me if I am wrong but 1.3 is 1year old.

Do you have an estimate date for 1.4 ?

By the way could you please summarise if a workaroud exist for 1.3 ?
Here is my syntax:
jq ".name.PS_Perm_Gen.peak-usage.init" < /tmp/localhost-localhost-9990-Memory

And, I've tried:
jq .["name.PS_Perm_Gen.peak-usage.init"] < /tmp/localhost-localhost-9990-Memory
error: Invalid numeric literal (while parsing '..')

and that:
jq .name.PS_Perm_Gen.["peak-usage"].init < /tmp/localhost-localhost-9990-Memory
error: syntax error, unexpected '[', expecting IDENT
.name.PS_Perm_Gen.[peak-usage].init

both does not work

greetings
Akram

[nicowilliams]

I didn't finish setting up build envs for Win64 in time and then business
travel intervened. I'll get to it soon, but not this week.

[ghost]

@akram jq .name.PS_Perm_Gen["peak-usage"].init is what you're looking for.

There are two syntaxes for element access: .foo and ["foo"]. Note the lack of leading dot in the second case. The second syntax allows for extraneous characters, the first one does not.

If you're using master, as well as in the future 1.4 version, ."foo" is a valid option as well. It also allows extraneous characters.

[hughmandeville]

Wrap hyphenated name in escaped quotation marks.

jq "[.[]|{\"my-field\"}]" test.json 

[
  {
    "my-field": "val1"
  },
  {
    "my-field": "val2"
  }
]

[ereyes01]

I think it's important to note that if you ever script the query string to jq, it would be unsafe in the generic case to not use the jq ".[$keyname]" notation if the names of your keys are not statically known/defined. A simple jq ".$keyname" would break if $keyname ever has a hyphen in it.

[ereyes01]

In fact, it seems like it would be more correct if, in your parser, you would prioritize matching key names over evaluating operators.

example: echo '{"key-10": 10}' | jq ".key-10"

Ideally would print "10"... and:

echo '{"key-10": 10}' | jq ".key-10-10"

would print "0". I say this of course, after having thought about all the possibilities for a full 5 minutes or so :-)

[dtolnay]

Then you have a problem if someone wants to subtract 10 from a key:

echo '{"key": 11, "key-10": 12}' | jq ".${KEY}-10"

This would require the even more confusing ".${KEY}-(10)" or "(.${KEY})-10".

[hexsel]

Just for posterity, the following seems to work:

jq .\"a-b\"

Seems like the shortest working query string with a hyphen.

[mcwumbly]

I ran into this issue today and this workaround in the comment above doesn't seem to work for me in jq 1.3. Anyone have any tips for what works today when strings have dashes in them?

[pkoppstein]

@mcwumbly - The short of it is that, given a JSON object, it's always safe to use the form: .[ $keyname ] where $keyname is a JSON string.

Note also that the fact that .X | .Y works for some X and Y does NOT always mean that .X.Y will work.

For example, .[0] | .["foo-bar"] is valid, but .[0].["foo-bar"] is not.

Alternatively:

$echo '[{ "foo-bar":"baz" }]' | jq '.[0][ "foo-bar" ]'
"baz"

[wtlangford]

I should also point out that jq-1.3 is quite old. jq-1.5 is the current
release, and jq-1.6 is hopefully released soon (pending mine or
@nicowilliams' schedules).

As for jq-1.3, is that the package provided by your system or by your
sysadmin? In the latter case, I'd recommend pestering them to upgrade you
to 1.5 (which is much nicer in a lot of ways, and should not have any
backwards incompatibilities). If the former... I'm going to guess you're on
Debian. ;)

On Tue, Jul 11, 2017 at 2:13 PM pkoppstein [e-mail] wrote:

Assuming the input is a JSON object, it's always safe to use the form: .[
$keyname ] where $keyname is a JSON string.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/stedolan/jq/issues/38#issuecomment-314527720, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ADQ4V3oOzLiAKHI6P03w7MAXr1rDQffqks5sM7stgaJpZM4AN_5L
.

[mcwumbly]

thanks @pkoppstein - that helps, and this works:

echo '{ "foo-bar":"baz" }' | jq '.[ "foo-bar" ]'

[pierreprinetti]

This works:

echo '[{ "foo-bar":"baz" }]' | jq '.[0]' | jq '.[ "foo-bar" ]'

# "baz"

This doesn't:

echo '[{ "foo-bar":"baz" }]' | jq '.[0].[ "foo-bar" ]'

# jq: error: syntax error, unexpected '[', expecting FORMAT or QQSTRING_START (Unix shell quoting issues?) at <top-level>, line 1:
# .[0].[ "foo-bar" ]
# jq: 1 compile error

[nicowilliams]

Instead, this does not seem to work:

echo '[{ "foo-bar":"baz" }]' | jq '.[0].[ "foo-bar" ]'

You don't need that second period. Use jq '.[0]["foo-bar"]'.

That is, to chain index operations you must not use a period between the index operators. Only the first index operator needs a period (so as to distinguish it from the array constructor syntax).

We should probably make it so that extra period is not a syntax error, as this is a bit of an FAQ.

(Replying by email, so I think the formatting won't show correctly. But I'll set the MIME type to see if I can get it to work.)

[pkoppstein]

key="foo-bar"; echo '[{ "foo-bar":"baz" }]' | jq '.[0].[ "$key" ]'

There are two problems with the above. First, the abbreviation .[0].["KEYNAME"] is not (currently) supported. The second has to do with the relationship between the shell or environment variable $key and the jq variable $key.

In a nutshell, here is what is recommended:

key="foo-bar"; echo '[{ "foo-bar":"baz" }]' | jq --arg key "$key" '.[0]|.[ $key ]'

For future reference, please ask usage questions at stackoverflow.com with the jq tag: https://stackoverflow.com/questions/tagged/jq

[dhjw]

If you need to pass a variable containing a string with a dash and save it in a variable in a bash script...

result=$(jq --arg a "$query" '.[$a]' -r some.json)