Jq: Allow "raw" output of arrays

[arodland]

When using jq -r, if the result is an array, then assume it's an array of strings, and print them, one per line, without quotes, without commas, and without a leading [ or trailing ].

The goal is that this:

echo '{"results":[{"id":1},{"id":2},{"id":3}]}' | jq -r '.results | map("ID is \(.id)")'

Will output

ID is 1
ID is 2
ID is 3

instead of

[
  "ID is 1",
  "ID is 2",
  "ID is 3"
]

which I think would make jq -r that much more useful for interacting with non-JSON tools. In particular it would let you generate tab-separated files.

[lluchs]

Just append | .[] to unwrap the array at the end of your expression:

$ echo '{"results":[{"id":1},{"id":2},{"id":3}]}' | jq -r '.results | map("ID is \(.id)") | .[]'
ID is 1
ID is 2
ID is 3

[arodland]

That's pretty cool. Maybe work it into the docs so that it's clear to other people that you can do that?

[ruckus]

I, too, am trying to accomplish this. I would like to product valid JSON output from a JSON source (essentially being a pipeline so I can feed JSON cleaned up by jq into another program). I'm only able to have jq spit out each object per line without wrapping quotes and a final wrapping [ and ].

cat zephyr.json  | jq -r '.[] | select(.["location-prefix"]=="AC") | {code: .["product-code"], uom: .["um-code"]}'

Returns

{
  "uom": "CS12",
  "code": "VWSEZUGABI101"
}
{
  "uom": "CS12",
  "code": "VWSEZUGARI071"
}

Ideally it would be like:

[

{
  "uom": "CS12",
  "code": "VWSEZUGABI101"
},
{
  "uom": "CS12",
  "code": "VWSEZUGARI071"
}
]

Anyway to accomplish this that I am missing? Thanks.

[nicowilliams]

Add '|[.]' to the end of your jq program.

On Mon, Nov 25, 2013 at 6:33 PM, Cody Caughlan [email protected]:

I, too, am trying to accomplish this. I would like to product valid JSON
output from a JSON source (essentially being a pipeline so I can feed JSON
cleaned up by jq into another program). I'm only able to have jq spit out
each object per line without wrapping quotes and a final wrapping [ and ].

cat zephyr.json | jq -r '.[] | select(.["location-prefix"]=="AC") | {code: .["product-code"], uom: .["um-code"]}'

Returns

{
"uom": "CS12",
"code": "VWSEZUGABI101"}{
"uom": "CS12",
"code": "VWSEZUGARI071"}

Ideally it would be like:

[
{
"uom": "CS12",
"code": "VWSEZUGABI101"},{
"uom": "CS12",
"code": "VWSEZUGARI071"}]

Anyway to accomplish this that I am missing? Thanks.

—
Reply to this email directly or view it on GitHubhttps://github.com/stedolan/jq/issues/189#issuecomment-29257221
.

[ruckus]

Trying that - but just results in each “entry” being wrapped in []:

cat zephyr.json | jq -r -a -c '.[] | select(.["location-prefix"]=="AC") | {code: .["product-code"], uom: .["um-code"]} | [.]’

Yields:

[{"uom":"CS12","code":"VWSEZUGABI101"}]
[{"uom":"CS12","code":"VWSEZUGARI071”}]

On Nov 25, 2013, at 7:41 PM, Nico Williams [email protected] wrote:

Add '|[.]' to the end of your jq program.

On Mon, Nov 25, 2013 at 6:33 PM, Cody Caughlan [email protected]:

I, too, am trying to accomplish this. I would like to product valid JSON
output from a JSON source (essentially being a pipeline so I can feed JSON
cleaned up by jq into another program). I'm only able to have jq spit out
each object per line without wrapping quotes and a final wrapping [ and ].

cat zephyr.json | jq -r '.[] | select(.["location-prefix"]=="AC") | {code: .["product-code"], uom: .["um-code"]}'

Returns

{
"uom": "CS12",
"code": "VWSEZUGABI101"}{
"uom": "CS12",
"code": "VWSEZUGARI071"}

Ideally it would be like:

[
{
"uom": "CS12",
"code": "VWSEZUGABI101"},{
"uom": "CS12",
"code": "VWSEZUGARI071"}]

Anyway to accomplish this that I am missing? Thanks.

—
Reply to this email directly or view it on GitHubhttps://github.com/stedolan/jq/issues/189#issuecomment-29257221
.

—
Reply to this email directly or view it on GitHub.

[nicowilliams]

Oops, excuse me, sorry, try jq -s.

[ruckus]

Thanks for your help Nico but its not working with -s : actually I had tried that earlier when I was first getting started but because my input JSON source is itself a JSON array I couldn’t get it to work - see the error below. But you might be on to something by using -s , I just need to figure out how to convert my query / filter to support a JSON input array?

$ cat zephyr.json | jq -r -a -c -s '.[] | select(.["location-prefix"]=="AC") | {code: .["product-code"], uom: .["um-code"]}'
jq: error: Cannot index array with array

On Nov 25, 2013, at 7:55 PM, Nico Williams [email protected] wrote:

Oops, excuse me, sorry, try jq -s.
—
Reply to this email directly or view it on GitHub.

[nicowilliams]

@arodland

echo '{"results":[{"id":1},{"id":2},{"id":3}]}' | ./jq -r '.results | map("ID is \(.id)")|.[]'

works for me.

[ruckus]

Here is a direct URL to a sample of my input file:

https://gist.github.com/ruckus/0d02700f248af3a00919/raw/3f83c738ab2a55caea7c09fc2fdd02db29ff1c94/gistfile1.txt

curl https://gist.github.com/ruckus/0d02700f248af3a00919/raw/3f83c738ab2a55caea7c09fc2fdd02db29ff1c94/gistfile1.txt | jq '.[] | {code: .["product-code"], uom: .["um-code"]} | .[]’

yields:

"CS12"
"BHWIBHCHSMV08"
"CS12"
"BHWIBHPNLEV08"
"CS12"
“BHWIBHPNSMV08"

and playing around with the final “ |.[]” doesn’t seem to help.

Thanks!

On Nov 25, 2013, at 8:04 PM, Nico Williams [email protected] wrote:

echo '{"results":[{"id":1},{"id":2},{"id":3}]}' | ./jq -r '.results | map("ID is (.id)")|.[]'
—
Reply to this email directly or view it on GitHub.

[nicowilliams]

@ruckus:

jq -s '[.[][] | {code: .["product-code"], uom: .["um-code"]}] '

works for me.

[nicowilliams]

Oops, didn't mean to close it (though I think it should be). If it's still a problem feel free to re-open.

[ruckus]

That works - thanks. I figured it was possible, just didn’t know the magic incantation. Thanks!

On Nov 25, 2013, at 8:21 PM, Nico Williams [email protected] wrote:

Oops, didn't mean to close it. If it's still a problem feel free to re-open.

—
Reply to this email directly or view it on GitHub.

[nicowilliams]

Actually, this makes me wonder... Consider this:

jq -s '.[][] | {code: .["product-code"], uom: .["um-code"]}|reduce . as $item ([]; . + [$item])'

which doesn't work. Why? Because reduce() reduces its individual inputs; there's no builtin function that corresponds to the code generated by the compile.c:gen_collect() function. But the bytecode generated by gen_collect() is very similar to the bytecode generated by gen_reduce(), so it makes sense that there ought to be a "collect" builtin that works like reduce, but applied to all its inputs rather than to each of its inputs. I.e., I'd like these two to be equivalent, but the second doesn't work:

jq '[.]'
jq 'collect . as $item ([]; . + $item)

or even just:

jq 'collect'

(since one could always reduce the output of collect, as one can reduce the output of [exp]).

In any case, for now the only way to collect items into an array is by placing square brackets around the expression producing the values to be collected. There's some degree to which lots of ways of expressing the same thing is desirable, and then again, not (c.f., Perl).

[Relequestual]

@nicowilliams: In any case, for now the only way to collect items into an array is by placing square brackets around the expression producing the values to be collected.

@nicowilliams Is this still the case? I'm trying to do somthing similar as a last action, because I don't know if i'll have an array or a singe item from the array in my given json.

[pkoppstein]

@Relequestual - You can, for example, now use reduce inputs .... (using -n instead of -s).

If that doesn’t answer your question, I would suggest asking it at stackoverflow.com with the jq tag:
https://stackoverflow.com/questions/tagged/jq

[Relequestual]

@pkoppstein Looks like -n is for building JSON data from scratch, aka null as an input. That's what the current docs tell me. I can't work out how that's related to what I asked. (I'm happy to be enlightened, but it doesn't sound like you have the time, and that's OK)

[pkoppstein]

The -n option is generally needed when using inputs. Some of the documentation about -n might be a bit misleading, or you might have misinterpreted it -- it's a bit tricky to explain, but input and inputs will (attempt to) read from STDIN.

[Relequestual]

From the docs:

--null-input/-n:

Don’t read any input at all! Instead, the filter is run once using null as the input. This is useful when using jq as a simple calculator or to construct JSON data from scratch.

If you're saying this will read input from STDIN, then yes this is totally missleading.

Looks like I missunderstood the OPs question here. I'm looking to wrap IN an array, in unwrap an existing array output. Oops! =/

[pkoppstein]

“Don’t read any input at all!” should be interpreted to mean “Don’t read any input at all except as explicitly specified.”