Jq: Long tab indent and alignment

Created on 19 May 2017  路  16Comments  路  Source: stedolan/jq

When I try...

docker run -it --rm -t devth/alpine-bench -n3 -v2 http://google.com/ | jq -R 'fromjson?'

I got...
image

Which is look not pretty to me, any hint?

Thanks

picture katopz
👍1

Most helpful comment

I pipe it through sponge from moreutils. Seems to do the trick:

docker exec [...] | sponge | jq .
picture jarpy on 17 Jan 2019
🎉2 👍2

All 16 comments

  1. Please try jq -M .

  2. If that doesn't help, and if your jq supports the --indent option, try: jq -M --indent 1 .

  3. If none of the above helps, please consider whether jq -c . is an acceptable workaround, and let us know what version number of jq you are using.

pkoppstein picture pkoppstein on 19 May 2017

@katopz Looks like your terminal wants \r\n line delimiters. What if you pipe the output of jq to unix2dos?

nicowilliams picture nicowilliams on 19 May 2017

@pkoppstein

  1. No luck
  2. jq -M --indent 1 .
    $ docker run -it --rm -t devth/alpine-bench -n3 -v2 http://google.com/ | jq -M --indent 1 . parse error: Invalid numeric literal at line 1, column 11
  3. jq -c .
    $ docker run -it --rm -t devth/alpine-bench -n3 -v2 http://google.com/ | jq -c . parse error: Invalid numeric literal at line 1, column 11
    FYI
$ jq --version
jq-1.5

@nicowilliams I'm on macOS now not sure about unix2dos

Thanks

katopz picture katopz on 19 May 2017

@katopz Can you post jq's input?

nicowilliams picture nicowilliams on 20 May 2017

@katopz -- Did you try the following?

jq -M -R --indent 1 'fromjson?'

jq -c -R 'fromjson?'
pkoppstein picture pkoppstein on 20 May 2017

@nicowilliams 

$ docker run -it --rm -t devth/alpine-bench -n3 -v2 http://google.com/
2017/05/24 07:39:01 Running: ab -n3 -v2 http://google.com/
2017/05/24 07:39:01 This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking google.com (be patient)...INFO: GET header == 
---
GET / HTTP/1.0
Host: google.com
User-Agent: ApacheBench/2.3
Accept: */*


---
LOG: header received:
HTTP/1.0 302 Found
Location: http://www.google.co.th/?gws_rd=cr&ei=lTglWeb1EYir0QT_kqegAw
Cache-Control: private
Content-Type: text/html; charset=UTF-8
P3P: CP="This is not a P3P policy! See https://www.google.com/support/accounts/answer/151657?hl=en for more info."
Date: Wed, 24 May 2017 07:39:01 GMT
Server: gws
Content-Length: 261
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Set-Cookie: NID=104=HSX4PNT0PP9jWO-BAzUk93BCDkgkzvjSfjn_Ii8QWBcWRllAXSCf7uHDU8S1sWZxYSGMsBWBRrCJ6lvf-q_gS7ScV9_NonGOeEOAkX8RtL_3uFE1JiDUe5cTm3aglQo-; expires=Thu, 23-Nov-2017 07:39:01 GMT; path=/; domain=.google.com; HttpOnly

<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>302 Moved</TITLE></HEAD><BODY>
<H1>302 Moved</H1>
The document has moved
<A HREF="http://www.google.co.th/?gws_rd=cr&amp;ei=lTglWeb1EYir0QT_kqegAw">here</A>.
</BODY></HTML>

WARNING: Response code not 2xx (302)
LOG: header received:
HTTP/1.0 302 Found
Location: http://www.google.co.th/?gws_rd=cr&ei=lTglWfevGMK40ASP8rjoDQ
Cache-Control: private
Content-Type: text/html; charset=UTF-8
P3P: CP="This is not a P3P policy! See https://www.google.com/support/accounts/answer/151657?hl=en for more info."
Date: Wed, 24 May 2017 07:39:01 GMT
Server: gws
Content-Length: 261
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Set-Cookie: NID=104=fQD_4gq3HjQz_p0N0KcZsUdeXOWR9g2-Kar1K2PKZ8joaP4MhXbsvUDdcBrmsOl9jI2fWoXWOXUN5-mqG05eC5Y84suC0zVRjOxCMZIIJ8Bp1GzX7dHLFfRiD1GDH98P; expires=Thu, 23-Nov-2017 07:39:01 GMT; path=/; domain=.google.com; HttpOnly

<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>302 Moved</TITLE></HEAD><BODY>
<H1>302 Moved</H1>
The document has moved
<A HREF="http://www.google.co.th/?gws_rd=cr&amp;ei=lTglWfevGMK40ASP8rjoDQ">here</A>.
</BODY></HTML>

WARNING: Response code not 2xx (302)
LOG: header received:
HTTP/1.0 302 Found
Location: http://www.google.co.th/?gws_rd=cr&ei=lTglWbfmHoXG0ASDxIuIDg
Cache-Control: private
Content-Type: text/html; charset=UTF-8
P3P: CP="This is not a P3P policy! See https://www.google.com/support/accounts/answer/151657?hl=en for more info."
Date: Wed, 24 May 2017 07:39:01 GMT
Server: gws
Content-Length: 261
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Set-Cookie: NID=104=NKmtmZ8TUbTDvpscCQq5rmVmeudItVFsyVTlmKN-IEQNEyltN9wlszn9M0BSQHM0bTbjVkO2LcOW1G407ZF2KG_Op8gf82IcJlS7ycTJrbqbnyrcJyEa4HlnOilu-Z3r; expires=Thu, 23-Nov-2017 07:39:01 GMT; path=/; domain=.google.com; HttpOnly

<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>302 Moved</TITLE></HEAD><BODY>
<H1>302 Moved</H1>
The document has moved
<A HREF="http://www.google.co.th/?gws_rd=cr&amp;ei=lTglWbfmHoXG0ASDxIuIDg">here</A>.
</BODY></HTML>

WARNING: Response code not 2xx (302)
..done


Server Software:        gws
Server Hostname:        google.com
Server Port:            80

Document Path:          /
Document Length:        261 bytes

Concurrency Level:      1
Time taken for tests:   0.316 seconds
Complete requests:      3
Failed requests:        0
Non-2xx responses:      3
Total transferred:      2685 bytes
HTML transferred:       783 bytes
Requests per second:    9.49 [#/sec] (mean)
Time per request:       105.367 [ms] (mean)
Time per request:       105.367 [ms] (mean, across all concurrent requests)
Transfer rate:          8.30 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        7    8   0.8      9       9
Processing:    96   97   1.0     98      98
Waiting:       95   97   1.4     98      98
Total:        104  105   1.1    106     106
WARNING: The median and mean for the initial connection time are not within a normal deviation
        These results are probably not that reliable.
WARNING: The median and mean for the processing time are not within a normal deviation
        These results are probably not that reliable.

Percentage of the requests served within a certain time (ms)
  50%    105
  66%    105
  75%    106
  80%    106
  90%    106
  95%    106
  98%    106
  99%    106
 100%    106 (longest request)

2017/05/24 07:39:01 Matched stat line [Connect:        7    8   0.8      9       9 Connect 7 8 0.8 9 9]
2017/05/24 07:39:01 Matched stat line [Processing:    96   97   1.0     98      98 Processing 96 97 1.0 98 98]
2017/05/24 07:39:01 Matched stat line [Waiting:       95   97   1.4     98      98 Waiting 95 97 1.4 98 98]
2017/05/24 07:39:01 Matched stat line [Total:        104  105   1.1    106     106 Total 104 105 1.1 106 106]
2017/05/24 07:39:01 Matched percentline [  50%    105 50 105]
2017/05/24 07:39:01 Matched percentline [  66%    105 66 105]
2017/05/24 07:39:01 Matched percentline [  75%    106 75 106]
2017/05/24 07:39:01 Matched percentline [  80%    106 80 106]
2017/05/24 07:39:01 Matched percentline [  90%    106 90 106]
2017/05/24 07:39:01 Matched percentline [  95%    106 95 106]
2017/05/24 07:39:01 Matched percentline [  98%    106 98 106]
2017/05/24 07:39:01 Matched percentline [  99%    106 99 106]
2017/05/24 07:39:01 Matched percentline [ 100%    106 100 106]
{"connect":{"max":9,"mean":8,"median":9,"min":7,"stdDev":0.800000011920929},"percentiles":{"100":106,"50":105,"66":105,"75":106,"80":106,"90":106,"95":106,"98":106,"99":106},"processing":{"max":98,"mean":97,"median":98,"min":96,"stdDev":1},"total":{"max":106,"mean":105,"median":106,"min":104,"stdDev":1.100000023841858},"waiting":{"max":98,"mean":97,"median":98,"min":95,"stdDev":1.399999976158142}}

Only last line is what I need, maybe I should ask to silent debug ;o

@pkoppstein
image

Thanks

katopz picture katopz on 24 May 2017 
$ jq -M -R --indent 1 fromjson
{"connect":{"max":9,"mean":8,"median":9,"min":7,"stdDev":0.800000011920929},"percentiles":{"100":106,"50":105,"66":105,"75":106,"80":106,"90":106,"95":106,"98":106,"99":106},"processing":{"max":98,"mean":97,"median":98,"min":96,"stdDev":1},"total":{"max":106,"mean":105,"median":106,"min":104,"stdDev":1.100000023841858},"waiting":{"max":98,"mean":97,"median":98,"min":95,"stdDev":1.399999976158142}}
{
 "connect": {
  "max": 9,
  "mean": 8,
  "median": 9,
  "min": 7,
  "stdDev": 0.800000011920929
 },
 "percentiles": {
  "100": 106,
  "50": 105,
  "66": 105,
  "75": 106,
  "80": 106,
  "90": 106,
  "95": 106,
  "98": 106,
  "99": 106
 },
 "processing": {
  "max": 98,
  "mean": 97,
  "median": 98,
  "min": 96,
  "stdDev": 1
 },
 "total": {
  "max": 106,
  "mean": 105,
  "median": 106,
  "min": 104,
  "stdDev": 1.100000023841858
 },
 "waiting": {
  "max": 98,
  "mean": 97,
  "median": 98,
  "min": 95,
  "stdDev": 1.399999976158142
 }
}

Something is odd about your terminal or environment. I don't know what it is.

nicowilliams picture nicowilliams on 26 May 2017

@nicowilliams I don't think you pipe from docker output.

katopz picture katopz on 6 Jun 2017

@nicowilliams https://github.com/nicowilliams I don't think you pipe
from docker output.

Indeed. Attach/quote that output please.

>

nicowilliams picture nicowilliams on 12 Jun 2017

To answer my own question, I've to add 2>/dev/null to redirect unwant output.

docker run -it --rm -t devth/alpine-bench -n3 -v2 http://google.com/ | jq -R 'fromjson?' 2>/dev/null

image

katopz picture katopz on 13 Jun 2017

I'm getting the same issue now. I tried your solution, but it's not working for me 馃槥
Can somebody help me please?

I'm running Linux Debian with jq-1.5-1-a5b5cbe

output

mujz picture mujz on 20 Dec 2018

I pipe it through sponge from moreutils. Seems to do the trick:

docker exec [...] | sponge | jq .
jarpy picture jarpy on 17 Jan 2019
🎉2 👍2

FWIW I was trying to pass output from a following kubectl (kubectl logs -f) command through an intermediate sed -u unbuffering script to tidy the output and on to jq.
Without the intermediary script jq works fine. With the sed -u script output didn't show up in the terminal presumably due to buffering.

In an attempt to fix it I ran via script as suggested here: https://unix.stackexchange.com/a/61833/49416

At this point I started getting the output described in this issue, and none of the suggested fixes work.
So for a long running process you can have non-following filtered-via-intermediary-sub-process input, or following unbuffered direct input, but not filtered-by-subprocess and unbuffered input.

Running jq-1.6 on mac High Sierra 10.13.6 with sponge obtained from brew install moreutils (moreutils: stable 0.63)

WaffleSouffle picture WaffleSouffle on 30 Aug 2019

Getting the same behavior when using a fish function:

function lncli --description 'Expose lncli from Docker container'
    docker-compose -p box exec lnd lncli $argv | jq .
end

MacOS 2019-11-22 at 10 04 33

meeDamian picture meeDamian on 22 Nov 2019

also having this issue with docker on osx.

./scripts/start-local-docker.sh | jq -R -r '. as $line | try fromjson catch $line'

image

jasonchester picture jasonchester on 19 Feb 2020

Drop the -t option so docker does not allocate a pseudo-tty.

pkocks picture pkocks on 30 Nov 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

kaihendry picture kaihendry  路  4Comments
tbelaire picture tbelaire  路  4Comments
thelonious picture thelonious  路  4Comments
lbrader picture lbrader  路  3Comments
thedward picture thedward  路  3Comments