Clickhouse: Clickhouse fails to import CSV from postgres pipe

Created on 29 Apr 2019  路  5Comments  路  Source: ClickHouse/ClickHouse

Description
CH fails to import CSV data. Maybe a parser error or something similar.

Reproduction

  • CH version: 19.4.3.11
  • CH:
    create table crash_test (x String, y String, z String) engine = Log();
  • PostgreSQL:
    create table crash_test (x text, y text, z text);
    insert into crash_test values ('1st', '''2nd', '3rd');
    COPY (select * from crash_test) TO PROGRAM 'clickhouse-client --query="INSERT INTO crash_test FORMAT CSV"' delimiter ',' CSV;

Expected behavior
Correct import of string with apostrophe to clickhouse and if not, message in the clickhouse-server.err.log (now it's completely silent on the clickhouse side).

Message in postgresql console after command
[2019-04-29 15:26:30] [38000] ERROR: program failed "clickhouse-client --query="INSERT INTO crash_test FORMAT CSV"" [2019-04-29 15:26:30] Description: child process fail with return code 27

question

All 5 comments

try force quote

COPY (select * from crash_test) TO PROGRAM 'clickhouse-client --query="INSERT INTO crash_test FORMAT CSV"' with csv force quote *;

no force quote

| => cat aaa
1st,'2nd,3rd

| => cat aaa|clickhouse-client -q 'INSERT INTO crash_test FORMAT CSV'
Code: 27. DB::Exception: Cannot parse input: expected , at end of stream.: (at row 1)

force quote


| => cat aab
"1st","'2nd","3rd"
| => cat aab|clickhouse-client -q 'INSERT INTO crash_test FORMAT CSV'

ah, another option by CH --format_csv_allow_single_quotes

cat aaa|clickhouse-client --format_csv_allow_single_quotes 0 -q 'INSERT INTO crash_test FORMAT CSV'

ah, another option by CH --format_csv_allow_single_quotes

cat aaa|clickhouse-client --format_csv_allow_single_quotes 0 -q 'INSERT INTO crash_test FORMAT CSV'

thank you, both answers fix the issue. So it's not a bug. But im not sure if other symbols can also crash parser if we escape apostrophe.

CSV parser works as expected
https://clickhouse.yandex/docs/en/interfaces/formats/#csv

Both double and single quotes are supported.

Check TSV https://clickhouse.yandex/docs/en/interfaces/formats/#tabseparated

Was this page helpful?
0 / 5 - 0 ratings