dbt

dbt turns warehouse SQL files into a dependency graph of models, tests, documentation, and build commands. It is most useful in ELT systems where raw data lands first and reviewed transformations publish marts in a data-warehouse.

A dbt model

A dbt model is a select statement saved as a file. dbt materializes it as a view, table, incremental table, or ephemeral query depending on configuration. This artifact defines a daily order fact and tests the grain:

-- models/marts/fct_orders.sql
{{ config(materialized="table") }}
 
select
  order_id,
  customer_id,
  date(order_ts) as order_date,
  amount_cents
from {{ ref("stg_orders") }}
where status = 'paid'

# models/marts/schema.yml
models:
  - name: fct_orders
    description: "One row per paid order."
    columns:
      - name: order_id
        data_tests:
          - not_null
          - unique
      - name: amount_cents
        data_tests:
          - not_null

The ref("stg_orders") call creates graph dependency and lets dbt build upstream staging before the mart. Tests are not observability decoration: they are blocking data-quality contracts for downstream dashboards and feature-pipelines.

Architecture

dbt commonly owns the transform layer inside ETL and ELT: sources describe raw tables, staging models clean names and types, intermediate models encode reusable joins, and marts publish dimensional-modelling facts and dimensions. Airflow may schedule dbt jobs, but dbt should own SQL dependencies and generated data-lineage for models.

Failure modes

ref dependencies are reliable only when teams avoid hard-coded database names in model SQL. Incremental models can drift from full-refresh behavior if late updates are not merged correctly. Tests that check only not_null miss semantic regressions such as currency changes or redefined statuses.

References

• dbt documentation: SQL models
• dbt documentation: About data tests property
• dbt documentation: Sources

Nav

Section — Data Engineering

← Airflow Data Quality →