Skip to main content

Models Are SELECTs, Built in Ref Order

Level 8: Batch Pipelines & Orchestrationmedium28 minmodels as SELECTsref dependenciesmaterializationCREATE TABLE ASDROP-and-rebuild idempotencyrecursive CTEtopological reasoning

Materialize a staging-to-mart model chain in the order its ref graph dictates, and derive that order from the graph itself with a recursive CTE.

A model is a SELECT with a name

Every junior data engineering posting that says "dbt experience" is asking about one workflow, and the workflow is simpler than the tool. A model is a single SELECT statement saved in a file. The file name is the table name. That is the whole idea.

-- models/staging/stg_orders.sql
{{ config(materialized='table') }}

SELECT order_id, customer_id, status, amount_usd, ordered_at
FROM {{ source('shop', 'raw_orders') }}
WHERE status <> 'cancelled'

You never write CREATE TABLE in that file. The tool wraps your SELECT in the right DDL and runs it. That wrapping is called materialization. The four built-in ones are view (the default), table, incremental, and ephemeral. The table materialization is a full refresh: drop whatever is there, create the table from the query, and it is the one you write by hand in this module.

ref() is the edge, and the tool reads the graph

A model that reads another model does not name the table. It calls ref():

-- models/marts/fct_customer_revenue.sql
SELECT
  c.customer_id, c.name, c.region,
  COUNT(o.order_id)             AS orders,
  ROUND(SUM(o.amount_usd), 2)   AS revenue_usd
FROM {{ ref('stg_customers') }} c
JOIN {{ ref('stg_orders') }} o ON o.customer_id = c.customer_id
GROUP BY c.customer_id, c.name, c.region

Each ref() is one edge in a dependency graph. The tool parses every model, collects the edges into a graph, sorts it topologically, and builds in that order. That is the honest one-line answer to "what does dbt actually do": it materializes SELECTs in dependency order and runs tests. Nothing about the answer is vendor-specific.

The naming canon is worth memorizing because reviewers read it as a signal:

  • stg_* (staging) is one model per source table. Rename, cast, filter out garbage. No joins.
  • int_* (intermediate) is optional glue when one mart needs several messy steps.
  • fct_* and dim_* (marts) are the business-facing tables people query.

One more model rounds out this lesson's graph, and it is the interesting one. mart_customer_health reads fct_customer_revenue, and because that fact joins orders and so loses every customer who has never ordered, it also reads stg_customers directly. Its two parents therefore sit at different distances from the sources, which is the case a build-order query has to get right.

Order of evaluation
  1. raw_orders / raw_customerssources, landed by ingestion
  2. stg_orders / stg_customersone SELECT per source: filter, rename, cast
  3. fct_customer_revenuejoins both staging models, one row per customer
  4. mart_customer_healthreads the fact and stg_customers, so its parents sit at two different depths
The ref graph for this lesson. Nothing at a later stage can be built before the stage it reads.

Full refresh is idempotent by construction

In this module you are the tool, so you write the DDL yourself, and the shape is always the same pair:

DROP TABLE IF EXISTS stg_orders;
CREATE TABLE stg_orders AS
SELECT order_id, customer_id, status, amount_usd, ordered_at
FROM raw_orders
WHERE status <> 'cancelled';

Module 8.2 proved idempotency by running a script twice and demanding the same end state. A full refresh passes that test for free: the DROP throws away the previous run before the CREATE rebuilds it, so run two lands on exactly what run one produced.

Common mistake: dropping the DROP TABLE IF EXISTS line. It looks harmless because a rebuild without it does not duplicate rows the way a naive INSERT does. What it does instead is error with "table stg_orders already exists" and abort the rest of your script, leaving the marts holding yesterday's data while the job log says the failure was in staging. Same root cause as a duplicating load (the script assumed a clean slate), different and more confusing symptom.

Interview nuance: "what does dbt actually do" is answered in one sentence: it compiles your SELECT files into DDL, resolves ref() into a dependency graph, builds in topological order, and runs the tests. Candidates who reach for the CLI flags first, and candidates who cannot say it without the tool's name in the sentence, both read as tool users rather than engineers.

On a real platform this differs. Here you write the DROP and CREATE TABLE AS yourself and read the graph out of a model_refs table. On a real project you write only the SELECT, the edges come from ref() calls the parser finds in your SQL, and the graph lives in a compiled manifest.json that also drives docs, lineage, and --select state:modified+. dbt Labs rebuilt that parser in Rust for the Fusion engine (roughly 30x faster parsing, plus column-level lineage) and Fivetran and dbt Labs merged in June 2026. The workflow in this lesson did not change.

Sample data for this example
CREATE TABLE model_refs (
  model     TEXT,   -- the model being built
  ref_model TEXT    -- the model or source it reads
);
INSERT INTO model_refs (model, ref_model) VALUES
  ('stg_orders',           'raw_orders'),
  ('stg_customers',        'raw_customers'),
  ('fct_customer_revenue', 'stg_orders'),
  ('fct_customer_revenue', 'stg_customers'),
  ('mart_customer_health', 'fct_customer_revenue'),
  ('mart_customer_health', 'stg_customers');
Worked example (SQL)
-- The manifest, one row per ref() edge: 'model' reads 'ref_model'.
-- A model whose name never appears in the model column is a source, so it is built first.
SELECT model, ref_model
FROM model_refs
ORDER BY model, ref_model;

Apply

Your turn

The task this lesson builds to.

Write a script that builds three tables from raw_orders and raw_customers, in an order that satisfies the model_refs edges between them, and that is safe to run twice:

  • stg_orders(order_id, customer_id, status, amount_usd, ordered_at), every raw_orders row whose status is not 'cancelled'.
  • stg_customers(customer_id, name, region), every raw_customers row.
  • fct_customer_revenue(customer_id, name, region, orders, revenue_usd), one row per customer, joining the two staging models: orders is that customer's count of staged orders and revenue_usd is ROUND(SUM(amount_usd), 2) over them.

Use DROP TABLE IF EXISTS x; followed by CREATE TABLE x AS SELECT ... for each model, which is the table materialization. model_refs says fct_customer_revenue reads both staging models, so it cannot be built first. The graph's fourth model, mart_customer_health, is not part of this build. The grader runs your script twice and compares all three tables.

5 hints and 6 automated checks are waiting in the workspace.

Practice

Make it stick

A second problem on the same idea, plus 3 bonus drills.

Write a query that returns every model in model_refs with its build depth, as (model, depth), shallowest first, over model_refs(model, ref_model).

A model that nothing else feeds is a source and sits at depth 0. Any other model sits one level deeper than the deepest thing it reads, so a model with two parents at different depths takes the larger one. Alias the columns exactly model and depth, and break ties within a depth by model ascending. This is the topological sort the tool does for you before it builds anything.

4 hints and 1 automated check are waiting in the workspace.