get_relation method (https://docs.getdbt.com/docs/writing-code-in-dbt/jinja-context/adapter/#get_relation) returns None when provided with database, schema and table name of a Redshift external table (AWS Spectrum).
get_relation on that external table. Should return a regular relation as it does for native Redshift tables, e.g. an output like analytics.spectrum_schema.mytable instead of None
### Screenshots and log output
-
Which database are you using dbt with?
The output of dbt --version:
0.16.0
The operating system you're using:
MacOS Mojave
The output of python --version:
3.6.2
### Additional context
-
Thanks for the report @tomekzbrozek!
Looks like we use the Postgres code to query the information schema on redshift: https://github.com/fishtown-analytics/dbt/blob/dev/octavius-catto/plugins/redshift/dbt/include/redshift/macros/adapters.sql#L154
but redshift provides a totally different set of tables for fetching this info: https://docs.aws.amazon.com/redshift/latest/dg/r_SVV_EXTERNAL_TABLES.html
I think we'd need to copy the "base" pg implementation into the redshift plugin, then update the code to also fetch external tables too.
thank you, yup I also used SVV_EXTERNAL_TABLES as a workaround to the issue reported here, pasting below to give a flavour.
({{ schema }} and {{ table }} vars are defined earlier in the body of my macro that serves a different purpose, I basically needed to know if the external table exists of not, this is why I came across this issue :) )
-- check_if_external_table_exists returns 1 if a given external table exists, 0 if doesn't
{% set check_if_external_table_exists %}
SELECT
COUNT(*) AS if_exists_flag
FROM
SVV_EXTERNAL_TABLES
WHERE
schemaname || '.' || tablename = '{{ schema }}.{{ table }}'
{% endset %}
-- the loop below fetches results of check_if_external_table_exists:
-- 0 if table doesn't exist, 1 if table exists, 2 for other (erroneous?) cases
-- more context on the `execute` Jinja variable:
-- https://docs.getdbt.com/docs/writing-code-in-dbt/jinja-context/execute
{% if execute %}
{% set if_exists_flag = run_query(check_if_external_table_exists).columns[0].values()[0] | int %}
{% else %}
{% set if_exists_flag = 2 | int %}
{% endif %}
Most helpful comment
Thanks for the report @tomekzbrozek!
Looks like we use the Postgres code to query the information schema on redshift: https://github.com/fishtown-analytics/dbt/blob/dev/octavius-catto/plugins/redshift/dbt/include/redshift/macros/adapters.sql#L154
but redshift provides a totally different set of tables for fetching this info: https://docs.aws.amazon.com/redshift/latest/dg/r_SVV_EXTERNAL_TABLES.html
I think we'd need to copy the "base" pg implementation into the redshift plugin, then update the code to also fetch external tables too.