Clickhouse: Incorrect size of index granularity expect mark 1080 totally have marks 1080

Created on 22 May 2019  路  2Comments  路  Source: ClickHouse/ClickHouse

Describe the bug
Querying failed, errors occur in system log.

How to reproduce

  1. Version: 19.6.2.11
  2. Interfaces used: clickhouse-client, JDBC (DataGrip)
  3. Data in table were collected for 2 months.
  4. Table contains up to 10 duplicates by sorting key and contains total 1 000 000 000 rows.
  5. After updating from 19.4.3 to 19.6.2.11 we started to get errors in system log and while selecting data, but these errors are inconstant. Sometimes queries are passed.
  6. Failed SQL query is attached.

For table description and parts description see attached files.
table.txt
parts.txt

Expected behavior
Data selected without any errors.

Errors in system log:
"Incorrect size of index granularity expect mark 1080 totally have marks 1080"
1.log

Errors while selecting:
"Code: 9, e.displayText() = DB::Exception: Size of filter (1944) doesn't match size of column (1948) (version 19.6.2.11 (official build))"
failed_query.txt - now that query is working.

Additional information:
New data is inserting using next command:

curl --silent --show-error --upload-file "${FILE}" "${CLICKHOUSE_URL}?query=INSERT%20INTO%20${CLICKHOUSE_TARGET_TABLE}%20(order,uid,login,symbol,digits,cmd,volume,open_time,state,open_price,sl,tp,close_time,gw_volume,expiration,reason,conv_reserv,conv_rates,commission,commission_agent,storage,close_price,profit,taxes,magic,comment,gw_order,activation,gw_open_price,gw_close_price,margin_rate,timestamp,api_data,unused_unk,server,backup_timestamp)%20FORMAT%20JSONEachRow"

Also at the same time first part of query worked well:

SELECT server,
       order,
       toDate(open_time) AS date,
       (volume / 100) AS volume_lot,
       0 AS entry
FROM dbname.mytablename
WHERE 1 = 1
  AND indexHint(server IN ('anotherserver', 'anotherserver2', 'anotherserver3', 'anotherserver4', 'anotherserver5', 'anotherserver6', 'anotherserver7', 'anotherserver10', 'anotherserver11'))
  AND toDate(backup_timestamp) >= toDate('2019-04-01')
  AND toStartOfMonth(toDate(open_time)) = '2019-04-01'
  AND cmd IN (0, 1)
ORDER BY backup_timestamp DESC
LIMIT 1 BY server, order

And query with UNION ALL didn't work.

(SELECT server,
       order,
       toDate(open_time) AS date,
       (volume / 100) AS volume_lot,
       0 AS entry
FROM dbname.mytablename
WHERE 1 = 1
  AND indexHint(server IN ('anotherserver', 'anotherserver2', 'anotherserver3', 'anotherserver4', 'anotherserver5', 'anotherserver6', 'anotherserver7', 'anotherserver10', 'anotherserver11'))
  AND toDate(backup_timestamp) >= toDate('2019-04-01')
  AND toStartOfMonth(toDate(open_time)) = '2019-04-01'
  AND cmd IN (0, 1)
ORDER BY backup_timestamp DESC
LIMIT 1 BY server, order)
UNION ALL
(SELECT server,
       order,
       toDate(close_time) AS date,
       (volume / 100) AS volume_lot,
       1 AS entry
FROM dbname.mytablename
WHERE 1 = 1
  AND indexHint(server IN ('anotherserver', 'anotherserver2', 'anotherserver3', 'anotherserver4', 'anotherserver5', 'anotherserver6', 'anotherserver7', 'anotherserver10', 'anotherserver11'))
  AND toDate(backup_timestamp) >= toDate('2019-04-01')
  AND toStartOfMonth(toDate(close_time)) = '2019-04-01'
  AND cmd IN (0, 1)
ORDER BY backup_timestamp DESC
LIMIT 1 BY server, order)
bug v19.6

Most helpful comment

I made a step-by-step "How to reproduce" repo.
https://github.com/coraxster/ch-case
Hope this will help to figure out the bug.
Feel free to ask some additional info.

All 2 comments

I made a step-by-step "How to reproduce" repo.
https://github.com/coraxster/ch-case
Hope this will help to figure out the bug.
Feel free to ask some additional info.

It reproduces with your test case.

Was this page helpful?
0 / 5 - 0 ratings