Dbt: Support overriding internal macros from root projects

Created on 7 Apr 2020  路  1Comment  路  Source: fishtown-analytics/dbt

Describe the bug

Users should be able to override plugin-specific macros from their root project.

Steps To Reproduce

{# macros/override.sql #}

{% macro postgres__get_columns_in_relation() %}

    {{ log("OVERRIDE", info=true) }}

{% endmacro %}
-- models/my_model.sql

{{ adapter.get_columns_in_relation(this) }}

I'd expect to see log output indicating that the override macro was being used. This does not work for macros named get_columns_in_relation or postgres__get_columns_in_relation.

I haven't done enough digging to know why this behavior would be different from something like create_table_as, but I suspect it has something to do with going through the context used by the adapter.get_* methods.

Expected behavior

Macros defined in a project should override dbt's core implementation, or the implementation provided by a dbt plugin.

The output of dbt --version:

0.16.0
bug

Most helpful comment

This happens because adapter.get_columns_in_relation doesn't take a manifest, so it doesn't pass a manifest to execute_macro, so execute_macro uses the "internal manifest", which only has macros from core.

We might be able to avoid this by parsing _all_ project macros up-front and building that into the "internal manifest". I feel a lot more comfortable with that path now that we've refactored manifest loading quite a bit.

The tricky part of that is the easy path here is probably going to involve running load_all_projects twice, which irks me.

>All comments

This happens because adapter.get_columns_in_relation doesn't take a manifest, so it doesn't pass a manifest to execute_macro, so execute_macro uses the "internal manifest", which only has macros from core.

We might be able to avoid this by parsing _all_ project macros up-front and building that into the "internal manifest". I feel a lot more comfortable with that path now that we've refactored manifest loading quite a bit.

The tricky part of that is the easy path here is probably going to involve running load_all_projects twice, which irks me.

Was this page helpful?
0 / 5 - 0 ratings