Aws-data-wrangler: Error when using a athena database name having a dash '-'

Created on 24 Jan 2020  路  4Comments  路  Source: awslabs/aws-data-wrangler

I am trying to execute an ALTER TABLE command and and when dbname has a '-' in it. The query errors.

db_name = 'glue-athena'
wr.athena.query(query = sql, database = db_name)

Below is the exception:
, QueryFailed("FAILED: ParseException line 1:8 missing EOF at '-' near 'glue'",)

question

Most helpful comment

Hi @dror-g,
The reason for the error is because Athena doesn't support dash in the databases and table names.

https://docs.aws.amazon.com/athena/latest/ug/glue-best-practices.html#schema-names

The only acceptable characters for database names, table names, and column names are lowercase letters, numbers, and the underscore character.

However Glue Data catalog accept the dash.

Athena is presto in the background and Glue is Hive, so there are some incompatibles between them.
The reason for Glue allow this kind of character is because you maybe is working with spark, hive, or any other tool.

I am going to add a comment about it

All 4 comments

Hi @sujayramaiah, thank you very much for this issue. It always help us to evolve the library.

Did you have tried this same query into the AWS Athena console?

Because I don't think that it is an issue with Wrangler it self. I've executed some tests here and I'm able to query against databases with "-" like a charm.

Seems to me that you probably are forgetting to quote the database name in your query.

E.g.

import awswrangler as wr

rows = wr.athena.query(query="SHOW TABLES IN `foo-boo`", database="foo-boo")

print(list(rows))

Closing by inactivity...

@igorborgest I'm getting the same error when using the Athena web console, no Wrangler involved.
(clicking on 3 dots near table name -> Load partitions).

Hi @dror-g,
The reason for the error is because Athena doesn't support dash in the databases and table names.

https://docs.aws.amazon.com/athena/latest/ug/glue-best-practices.html#schema-names

The only acceptable characters for database names, table names, and column names are lowercase letters, numbers, and the underscore character.

However Glue Data catalog accept the dash.

Athena is presto in the background and Glue is Hive, so there are some incompatibles between them.
The reason for Glue allow this kind of character is because you maybe is working with spark, hive, or any other tool.

I am going to add a comment about it

Was this page helpful?
0 / 5 - 0 ratings