Clickhouse: Large insert queries in version 19.3.x

[alniks]

In versions 19.3.x query parser breaks at large queries approximately 5000 lines (format CSV and TabSeparated). There were no such problems in 19.1.6 and before. Which setting influences this? There is no information in Changelog for version 19.3.3 and 19.3.4.

[filimonov]

What exactly do you mean by "parser brakes"? Can you provide some sample?

[alniks]

INSERT INTO events (source_id, contact_id, created_at, email, ip, latitude, longitude, type, url, user_agent, user_id, event_date) format CSV
8, 98765, '2015-07-02 06:00:02', '[email protected]', '', 0, 0, 1, '', '', 2,'2015-07-02'
... 5k similar lines
8, 104177, '2015-07-02 14:18:17', '[email protected]', '10.10.10.10', 43.15, -87.12, 2, '', 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/600.5.17 (KHTML, like Gecko)', 2,'2015-07-02'

the result:

ru.yandex.clickhouse.except.ClickHouseException: ClickHouse exception, code: 62, host: ip6-localhost, port: 8123; Code: 62, e.displayText() = DB::Exception: Syntax error: failed at position 1 (line 1, col 1): ,'2015-07-02' 8, 30690, '2015-07-02 06:09:55', '[email protected]', '11.11.11.11', 30.0, -27.0, 2, '', 'Mozilla. Expected one of: CREATE TABLE or ATTACH TABLE query, SELECT query, possibly with UNION, SHOW [TEMPORARY] TABLES|DATABASES [[NOT] LIKE 'str'], SELECT query, subquery, possibly with UNION, Query, CREATE, EXISTS, ATTACH, DETACH, DROP, SHOW, DESCRIBE, DESC, USE, SET, SELECT, WITH, KILL, SELECT subquery, list of elements, ALTER query, ALTER TABLE, Query with output, SHOW PROCESSLIST query, SHOW PROCESSLIST, RENAME query, RENAME TABLE, SET query, SYSTEM query, EXISTS or SHOW CREATE query, USE query, CHECK TABLE, DESCRIBE query, DROP query, TRUNCATE, INSERT query, INSERT INTO, KILL QUERY query, OPTIMIZE query, OPTIMIZE TABLE, SELECT query

The insert goes well, if I split the same file by 1k lines. The table is Buffer over MergeTree

[M1ha-Shvn]

Same problem. I've tried 19.3.3 and 19.3.4. On 19.1.8 it works fine
I use CollapsingMergeTree without Buffer.

[geldot]

We've been seeing this with all releases after 19.1 (so we're stuck pinned to that release until fixed). Using the HTTP interface and TSV import here.

Syntax errors arise at arbitrary points in the stream after the first few hundred rows with no Buffer, straight insert to MergeTree.

[geldot]

Possibly related to #4442 in the Kafka engine, pointing toward a bug introduced in lower level streaming classes

[alesapin]

Fix https://github.com/yandex/ClickHouse/pull/4454. Also can be avoided with setting (or query argument) max_query_size=1048576 regardless of query length.

[alesapin]

Fixed in 19.3.5