Jq: raw-input and slurp don't allow easy line oriented processing

[tommack]

I wish there was an option or combination of options that would let me replicate this behavior:

➜  echo "a\nb\nc" | jq -R '.' | jq -s '.'
[
  "a",
  "b",
  "c"
]

[pkoppstein]

What's wrong with:

$ echo "a\nb\nc" | jq -R 'split("\\n")'
[
  "a",
  "b",
  "c"
]

The usual situation (reading from a text file) is also covered:

$ echo $'a\nb\nc' | jq -R -s 'split("\n")'
[
  "a",
  "b",
  "c",
  ""
]

(Of course, one might want to exclude the last "".)

Similarly, if the input is valid JSON:

$ echo '"a\nb\nc"' | jq 'split("\n")'
[
  "a",
  "b",
  "c"
]

The case you describe is rather unusual - "neither fish nor fowl".

[tommack]

I realize splitting is an option too. The ask is for an option/switch to make this easier.

I don't consider this to be unusual. If I'm piping non-JSON data in from some other command, I either want to process as individual strings ("-R") or as an array of strings (for which there's no simple option). I see those two styles as way more useful than what you get with "-s -R".

[pkoppstein]

@tommack wrote:

I don't consider this to be unusual.

Just to be clear: the output of echo "a\nb\nc" is neither JSON ("fish") nor equivalent to what you'd find in an ordinary text file ("fowl" -- corresponding to the title of this thread, i.e. "line oriented processing").

Also, I agree that for line-oriented processing ("fowl"), the situation is imperfect -- because of the issue regarding the trailing "". But for the neither-fish-nor-fowl case, split("\\n") seems to me to be a perfect match.

[nicowilliams]

In master:

% printf 'a\nb\nc' | ./jq -Rn '[inputs]'
[
"a",
"b",
"c"
]

Here we use -n because we'll consume the inputs from the jq program
using the new builtin inputs, which outputs... you guessed it: the
inputs to jq.

[nicowilliams]

The idea is that we should grow the language to reduce pressure on the command-line.

Ideally jq wouldn't have any of the -nesR options. -e should always be the case and anyways, there should be an exit builtin (easy enough to arrange with error). -n should always be the case. -s can be done with [inputs]. -R should be done with something like rawinputs (which doesn't exist yet), but here we have the risk of mixing inputs and rawinputs, so I'm not sure yet that this is wise.

[tommack]

@nicowilliams, thanks for the explanation.

[dtolnay]

Closing because Nico's command jq -Rn '[inputs]' replicates the desired behavior.

[combinatorist]

Wait @nicowilliams, why would -n always be the case? Isn't the point of jq to "filter" json files? So, would you just always cat <file name> | jq ...?

[pkoppstein]

@combinatorist - Nico Williams was writing hypothetically, i.e. if he were designing jq from scratch, or if the goal was to eliminate as many command-line options as possible.