Jq: exit code does not reflect truncated JSON input

[cpitchford]

jq will exit 0 with truncated input rather than exit 4 (if no valid result was ever produced)

echo '{"key":"value"' > broken.json
echo '{"key":"value"}' > working.json

$ jq -e .key working.json  ; echo $?
"value"
0

$ jq -e .key broken.json  ; echo $? ;
0
0

It exits 0 seemingly without generating output

Validating the file with "." returns an empty array

$ jq -s . broken.json ; echo $?
[]
0

python -mjson.tool < broken.json
Expecting object: line 1 column 15 (char 14)

[wtlangford]

My jq (jq 1.5) does this.

 $ echo '{"key": "value"' | jq -e .key; echo $?
parse error: Unfinished JSON term at EOF at line 2, column 0
4

I'd make sure you're running jq 1.5. Your package manager probably has it.

[cpitchford]

Damnit, missed out the version I was using. I am using 1.5. I'm actually trying to use jq to validate a json file is actually valid json.. and that's failing:

echo '{"key":"value"' | jq  -e . ; echo $?
0
echo '{"key":"value"' | jq  -s . ; echo $?
[]
0
echo '{"key":"value"' | jq  . ; echo $?
0

jq --version
jq-1.5

[jcttrll]

I'm able to reproduce all three cases, also using jq 1.5. (The echo examples _do_ reproduce the issue for me; I switched to using a Bash here-string just to eliminate the pipe/subshell.) I added a couple cases (a single left-curly and a totally empty document) that also cause jq to return 0 inappropriately.

$> jq -e . <<<'{"key":"value"' ; echo $?
0

$> jq -s . <<<'{"key":"value"' ; echo $?
[]
0

$> jq . <<<'{"key":"value"' ; echo $?
0

$> jq . <<<'{' ; echo $?
0

$> jq . <<<'' ; echo $?
0

$> jq --version
jq-1.5

I'm on openSUSE Tumbleweed; here's the package info:

$> zypper info jq
Information for package jq:
---------------------------
Repository: repo-oss
Name: jq
Version: 1.5-3.1
Arch: x86_64
Vendor: openSUSE
Installed: Yes
Status: up-to-date
Installed Size: 93.5 KiB
Summary: A lightweight and flexible command-line JSON processor
Description: 
  A lightweight and flexible command-line JSON processor. jq is like sed for
  JSON data – you can use it to slice and filter and map and transform
  structured data with the same ease that sed, awk, grep and friends let
  you play with text.

Please let me know if you need further info.

[jcttrll]

Also just reproduced this in a Docker container running up-to-date Ubuntu 16.04:

$> jq --version
jq-1.5-1-a5b5cbe

$> apt-cache show jq
Package: jq
Priority: optional
Section: universe/utils
Installed-Size: 349
Maintainer: Ubuntu Developers <...>
Original-Maintainer: Simon Elsbrock <...>
Architecture: amd64
Version: 1.5+dfsg-1
Depends: libc6 (>= 2.14), libonig2 (>= 5.9.5)
Filename: pool/universe/j/jq/jq_1.5+dfsg-1_amd64.deb
Size: 144146
MD5sum: 2ca788a5d75815930d750ae083459ee8
SHA1: 1753652880182fb7e4c30bf51fc4a211ca34f968
SHA256: 599db72fcd173cafff17c78aa11f0f80df5e44ccf96371d48af7d16d0edc81af
Description: lightweight and flexible command-line JSON processor
Description-md5: 6f3a50ad5d3a200e13433cea785ae21c
Multi-Arch: foreign
Homepage: https://github.com/stedolan/jq
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Origin: Ubuntu

[jcttrll]

Also reproduced in a Docker container running up-to-date Fedora 24:

$> jq --version
jq-1.5

$> dnf info jq
Last metadata expiration check: 0:04:05 ago on Tue Aug  2 15:15:29 2016.
Installed Packages
Name        : jq
Arch        : x86_64
Epoch       : 0
Version     : 1.5
Release     : 3.fc24
Size        : 343 k
Repo        : @System
From repo   : fedora
Summary     : Command-line JSON processor
URL         : http://stedolan.github.io/jq/
License     : MIT and ASL 2.0 and CC-BY and GPLv3
Description : lightweight and flexible command-line JSON processor
            : 
            :  jq is like sed for JSON data \u2013 you can use it to slice
            :  and filter and map and transform structured data with
            :  the same ease that sed, awk, grep and friends let you
            :  play with text.
            : 
            :  It is written in portable C, and it has zero runtime
            :  dependencies.
            : 
            :  jq can mangle the data format that you have into the
            :  one that you want with very little effort, and the
            :  program to do so is often shorter and simpler than
            :  you'd expect.

This appears to be a widespread problem. Impact for people using jq to validate the structure of JSON documents is high. I encountered the problem "in the wild" when a script using jq for this purpose allowed an empty file to slip through as "valid" JSON.

[jcttrll]

Most of this appears to have been fixed in c4524da, back on 2015-09-21, for #951. Unfortunately, the jq-1.5 tag (from which various distros seem to have done their builds) sits at a5b5cbe, which is from 2015-08-17 and is 83 commits behind the fix.

master at this time (0b82185) still treats empty input (or all-whitespace input) as valid, which is probably expected behavior, so ignore that.

Question: What is the release process? Who is in charge of it, and how can we get the necessary tags in place so the various package maintainers in the various distros can update and pick up nearly a year's worth of bugfixes, etc.?

[mcg]

Bumping this as we where just bitten by this bug. Any chance of a release with the fix?