Clickhouse: Table function 's3' cannot access non-default region

[ikosyanenko]

I am trying to query parquets from S3 in a next way:
select * from s3('https://s3.eu-west-1.amazonaws.com/..., 'key', 'secret', 'Parquet', '...') LIMIT 10

But getting:
Received exception from server (version 20.3.7): Code: 499. DB::Exception: Received from localhost:9000. DB::Exception: Unable to parse ExceptionName: AuthorizationHeaderMalformed Message: The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'eu-west-1'.

[vadv]

also did not find a way to access AWS S3 in the non-default region

[filimonov]

It should be fixed in #11036
Can you try 20.5-prestable?

[vadv]

Can you try 20.5-prestable?

problem with a long build:

23:13
[123/7955] Building CXX
...
08:55
[6346/7955] Building CXX object

is there any pre-build? :)

[filimonov]

Of course - there are prestable and testing channels on both rpm & deb repos.

Deb
https://repo.clickhouse.tech/deb/prestable/main/
https://repo.clickhouse.tech/deb/testing/main/

Rpm
https://repo.yandex.ru/clickhouse/rpm/prestable/x86_64/
https://repo.yandex.ru/clickhouse/rpm/prestable/x86_64/

[vadv]

@filimonov

INSERT INTO FUNCTION s3('https://bucket.s3-ap-southeast-1.amazonaws.com/clickhouse-backup/envoy/2020/06/23/envoy.csv', '<key>', '<secret>', 'CSV', 'response_code String') VALUES


Received exception from server (version 20.5.1):
Code: 499. DB::Exception: Received from localhost:9001. DB::Exception: <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AuthorizationHeaderMalformed</Code><Message>The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'ap-southeast-1'</Message><Region>ap-southeast-1</Region><RequestId>X</RequestId><HostId>X</HostId></Error>.

[vadv]

2020.06.24 08:27:36.270452 [ 911 ] {df3a58dd-970f-407a-bc37-3935dcceb3f6} <Warning> AWSClient: AWSClient: If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.
2020.06.24 08:27:36.270478 [ 911 ] {df3a58dd-970f-407a-bc37-3935dcceb3f6} <Warning> AWSClient: AWSClient: Request failed, now waiting 12800 ms before attempting again.
2020.06.24 08:27:49.090006 [ 911 ] {df3a58dd-970f-407a-bc37-3935dcceb3f6} <Error> AWSClient: AWSClient: HTTP response code: 400
Exception name:
Error message: <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AuthorizationHeaderMalformed</Code><Message>The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'ap-southeast-1'</Message><Region>ap-southeast-1</Region><RequestId>X</RequestId><HostId>X</HostId></Error>
7 response headers:
connection : close
content-type : application/xml
date : Wed, 24 Jun 2020 08:27:48 GMT
server : AmazonS3
transfer-encoding : chunked
x-amz-id-2 : X=
x-amz-request-id : X
2020.06.24 08:27:49.090055 [ 911 ] {df3a58dd-970f-407a-bc37-3935dcceb3f6} <Warning> AWSClient: AWSClient: If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.
2020.06.24 08:27:49.090372 [ 911 ] {df3a58dd-970f-407a-bc37-3935dcceb3f6} <Error> executeQuery: Code: 499, e.displayText() = DB::Exception: <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AuthorizationHeaderMalformed</Code><Message>The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'ap-southeast-1'</Message><Region>ap-southeast-1</Region><RequestId>X</RequestId><HostId>X</HostId></Error> (version 20.5.1.3833 (official build))

[vadv]

if clickhouse uses the AWS SDK, why are AWS_ * env variables not being processed?

[vzakaznikov]

@excitoon

[rupertevill]

Any update on this?

[alijared]

Why can't the region be parsed as a parameter to the table function?

[alexey-milovidov]

Fix: https://github.com/ClickHouse/ClickHouse/pull/15646