JSONEachRow produces an integer overflow on loading numbers greater than MAX_INT (2147483647)
How to reproduce
CREATE TABLE json_load_test (
`idx` Int32
) ENGINE = MergeTree
PARTITION BY cityHash64(idx)
ORDER BY idx
SETTINGS index_granularity = 8192;
INSERT INTO json_load_test (idx)
VALUES (1);
-- OK
INSERT INTO json_load_test (idx)
VALUES (2147483647);
-- OK
INSERT INTO json_load_test (idx)
VALUES (2147483647 + 1);
-- SQL Error [321]: ClickHouse exception, code: 321, host: *, port: *; Code: 321, e.displayText() = DB::Exception: Expression returns value NULL, that is out of range of type Int32, at: 2147483647 + 1) (version 19.15.3.6)
SELECT *
FROM json_load_test
/*
idx
----------
2147483647
1
*/
INSERT INTO json_load_test FORMAT JSONEachRow
{"idx": 2147483648}
-- OK
/*
idx
-----------
2147483647
-2147483648
1
*/
INSERT INTO json_load_test FORMAT JSONEachRow
{"idx": 21474836445}
{"idx": 9283409723894723}
-- OK
/*
idx
-----------
1122347971
-2147483648
1
-35
2147483647
*/
Expected behavior
Exception like a Expression returns value NULL, that is out of range of type Int32
Not related to JSON. CH does overflow by performance reasons.
SELECT toInt8(2147483644)
ββtoInt8(2147483644)ββ
β -4 β
ββββββββββββββββββββββ
SELECT toInt32('2147483648')
ββtoInt32('2147483648')ββ
β -2147483648 β
βββββββββββββββββββββββββ
But why INSERT INTO ... FORMAT JSONEachRow overflow behaviour differs from INSERT INTO ... VALUES?
There is an explanation in the documentation?
streaming parser:
INSERT INTO json_load_test (idx) VALUES (2147483648);
INSERT INTO json_load_test FORMAT JSONEachRow {"idx": 2147483648}
SELECT * FROM json_load_test
ββββββββββidxββ
β -2147483648 β
βββββββββββββββ
ββββββββββidxββ
β -2147483648 β
βββββββββββββββ
not streaming parser
INSERT INTO json_load_test (idx) VALUES (2147483647 + 1);
Cannot insert NULL value into a column of type 'Int32' at: 2147483647 + 1);
+ -- turn off the streaming parser because it's the expression.
Most helpful comment
streaming parser:
not streaming parser
+-- turn off the streaming parser because it's the expression.