Clickhouse: JSONEachRow causes an Int32 overflow without any exception on 19.15.3.6

Created on 5 Nov 2019  Β·  3Comments  Β·  Source: ClickHouse/ClickHouse

JSONEachRow produces an integer overflow on loading numbers greater than MAX_INT (2147483647)

How to reproduce

CREATE TABLE json_load_test (
    `idx` Int32
) ENGINE = MergeTree 
  PARTITION BY cityHash64(idx) 
  ORDER BY idx 
  SETTINGS index_granularity = 8192;


INSERT INTO json_load_test (idx) 
VALUES (1);
-- OK

INSERT INTO json_load_test (idx) 
VALUES (2147483647);
-- OK

INSERT INTO json_load_test (idx) 
VALUES (2147483647 + 1);
-- SQL Error [321]: ClickHouse exception, code: 321, host: *, port: *; Code: 321, e.displayText() = DB::Exception: Expression returns value NULL, that is out of range of type Int32, at: 2147483647 + 1) (version 19.15.3.6)

SELECT *
FROM json_load_test
/*
idx       
----------
2147483647
         1
*/

INSERT INTO json_load_test FORMAT JSONEachRow
{"idx": 2147483648}
-- OK
/*
idx        
-----------
 2147483647
-2147483648
          1
 */

INSERT INTO json_load_test FORMAT JSONEachRow
{"idx": 21474836445}
{"idx": 9283409723894723}
-- OK
/*
idx        
-----------
 1122347971
-2147483648
          1
        -35
 2147483647
 */

Expected behavior

Exception like a Expression returns value NULL, that is out of range of type Int32

st-declined

Most helpful comment

streaming parser:

INSERT INTO json_load_test (idx)  VALUES (2147483648);
INSERT INTO json_load_test FORMAT JSONEachRow {"idx": 2147483648}

SELECT * FROM json_load_test

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€idx─┐
β”‚ -2147483648 β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€idx─┐
β”‚ -2147483648 β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

not streaming parser

INSERT INTO json_load_test (idx) VALUES (2147483647 + 1);
Cannot insert NULL value into a column of type 'Int32' at: 2147483647 + 1);

+ -- turn off the streaming parser because it's the expression.

All 3 comments

Not related to JSON. CH does overflow by performance reasons.

SELECT toInt8(2147483644)

β”Œβ”€toInt8(2147483644)─┐
β”‚                 -4 β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

SELECT toInt32('2147483648')

β”Œβ”€toInt32('2147483648')─┐
β”‚           -2147483648 β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

But why INSERT INTO ... FORMAT JSONEachRow overflow behaviour differs from INSERT INTO ... VALUES?

There is an explanation in the documentation?

streaming parser:

INSERT INTO json_load_test (idx)  VALUES (2147483648);
INSERT INTO json_load_test FORMAT JSONEachRow {"idx": 2147483648}

SELECT * FROM json_load_test

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€idx─┐
β”‚ -2147483648 β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€idx─┐
β”‚ -2147483648 β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

not streaming parser

INSERT INTO json_load_test (idx) VALUES (2147483647 + 1);
Cannot insert NULL value into a column of type 'Int32' at: 2147483647 + 1);

+ -- turn off the streaming parser because it's the expression.

Was this page helpful?
0 / 5 - 0 ratings