Dbt: Configurable node status in log output

Created on 22 Jun 2020  路  1Comment  路  Source: fishtown-analytics/dbt

Describe the feature

Users should be able to control the information that dbt includes in its standard log output.

Per @drewbanin in #2526:

  1. Provide a config option that accepts a templated string for the model status
  2. Provide a context that has variables in scope (like job, materialization, status, runtime, etc)

Describe alternatives you've considered

  • We continue to make these decisions for everyone
  • We define one-off config parameters that users can flip on or off in profiles.yml

Who will this benefit?

  • This is most immediately beneficial on BigQuery, for which pricing depends on information other than end-to-end query timing
  • I can imagine that other adapters will have their own attendant metadata they want to report

cc @bodschut @alepuccetti

cli enhancement

Most helpful comment

After reading through the discussion in #2526, I'd like to add to request an additional (small) bit of configuration (along with bytes_processed and slot_duration) to specify exact versus rounded row counts. In dbt 0.18.0, the logging output for row counts changed from exact values (like 18 426) to rounded values (like 18K). I understand the design decisions behind this shift, and this compressed format suits us most of the time. However, there are times when having an exact row count would be useful for debugging purposes.

Tagging @jars who also has a stake in this :)

>All comments

After reading through the discussion in #2526, I'd like to add to request an additional (small) bit of configuration (along with bytes_processed and slot_duration) to specify exact versus rounded row counts. In dbt 0.18.0, the logging output for row counts changed from exact values (like 18 426) to rounded values (like 18K). I understand the design decisions behind this shift, and this compressed format suits us most of the time. However, there are times when having an exact row count would be useful for debugging purposes.

Tagging @jars who also has a stake in this :)

Was this page helpful?
0 / 5 - 0 ratings

Related issues

jgillies picture jgillies  路  3Comments

drewbanin picture drewbanin  路  4Comments

bashyroger picture bashyroger  路  3Comments

drewbanin picture drewbanin  路  4Comments

chrisburrell picture chrisburrell  路  3Comments