"Transformations as Code" — dbt Fundamentals

Skim Card

One thing to remember: SQL that produces numbers people sign is code: give it modules, a dependency graph, tests in every build, and history it can’t lose.

• A model is one SELECT; ref() and source() give dbt the graph, so build order and selection come from the code, not from cron.
• Clean each source once in staging (re-sent baskets dropped in stg_pos_sale_lines); marts only join and aggregate.
• Run dbt build: unit tests before a model, data tests after, everything downstream skipped on failure. Every fact gets a grain test.
• A snapshot keeps history the source overwrites, but only as often as it runs: weekly runs lost a promotion version that priced 1,908 lines.
• On Trino and Iceberg, on_table_exists: replace makes each build one commit to the same table; earlier builds stay readable by time travel.
• The dbt-built star reproduced Part 5 exactly: UK Q3 net revenue £29,790,674.06; stock 91.1% as reported, 88.5% at close.
• dbt v2 has no Trino adapter and parses strictly: dbt-oss 2.0.8 rejected a misspelt key v1 ignored, and bundles DuckDB 1.5.4.

Use …	When …
dbt-core 1.12 + your engine’s adapter	v2 has no GA adapter for the engine (Trino), or the project relies on v1-only behaviour
dbt-oss 2.0	The adapter is GA in v2 and you need Apache 2.0; fix every strict-parse error first
dbt 2.0	You want its language server, linting and SQL comprehension, under its proprietary licence
A snapshot	The source overwrites rows and nothing else keeps the old versions
A model over raw history	The raw zone already keeps every version (daily extracts, a CDC history table)

Verified against: dbt-core 1.12.6, dbt-trino 1.11.0, dbt-duckdb 1.11.1, dbt-oss 2.0.8 (DuckDB 1.5.4 inside), dbt_utils 1.4.1, Trino 484, Apache Polaris 1.8.0, Iceberg format v2 tables, DuckDB 2.0.0 · vendor docs · 28 October 2026

Jump to: Concepts · Hands-On · Landscape · Design Drill

Story Opening

Since Part 4, Larkspur’s star schema has been a Python script running SQL files, in an order only Rohan knows, on one laptop. Farah’s question was short: “Which version of that code produced the Q3 number I signed?”

“Fair,” Rohan said. “I’ll put the SQL files in Git and schedule them with cron.”

“That’s how teams end up with 900 scripts,” Ingrid said. “You wouldn’t ship the order service as scripts and a crontab. Modules, a build that knows the dependencies, tests in every build, review. And someone has to keep the history the promotions tool throws away.”

The Quick Map

Concepts, with use cases

dbt compiles a folder of SQL SELECTs and YAML into a dependency graph and runs it on a warehouse or query engine; it moves no data itself. It is the T in ELT (Part 2).

Models, sources and ref()

A model is one SELECT in one file; dbt wraps it in the DDL for a view or table named after the file. A source declares a table dbt didn’t build (source('raw', 'pos_uploads')); ref('stg_stores') points at another model. From those dbt knows the graph before running anything: build order, what --select +fct_sales_line means (the model and everything upstream), what a change touches.

What breaks without it. Hard-coded table names run in cron’s order, always against production; a second environment means editing every file. Larkspur: 13 sources, 33 models. Elsewhere: a hospital group’s billing marts, where one changed staging model must rebuild exactly the reports that read it.

Layers: staging, intermediate, marts

Staging models are one per source: rename, cast, deduplicate. Intermediate models hold joins and business logic. Marts are what people query: facts and dimensions. Staging and intermediate go to Part 9’s silver namespace, marts to gold.

What breaks without it. Each report repeats the cleaning, slightly differently: three numbers, two answers each. Larkspur: the till re-send fix from Part 6 lives once, in stg_pos_sale_lines, and every fact inherits it. Elsewhere: a SaaS company defining “active subscription” once, upstream of finance’s and product’s dashboards.

Materialisations

How a model lands, chosen per model:

dbt also builds snapshots (below) and seeds: small CSVs kept in the repo and loaded as tables, such as an airline’s hand-kept airport-code list (Larkspur needs none: its country map derives from the stores). What breaks if you pick blind. All views, and every dashboard re-runs the chain; all tables, and every build rewrites rows that didn’t change. Larkspur: staging views, intermediate and mart tables; the big facts go incremental in Part 18. Elsewhere: an adtech team that keeps hourly aggregates incremental and the advertiser dimension table.

Tests: data tests and unit tests

A data test is a query that returns the rows breaking a rule; zero rows passes. Generic ones are declared in YAML (unique, not_null, relationships, accepted_values, plus packages); singular ones are a SELECT in tests/. A unit test (dbt 1.8+) feeds a model fixed input rows and compares its output with expected rows: it checks the logic, not today’s data.

What breaks without them. A join to a dimension with a gap silently loses lines; a fan-out double-counts revenue. Neither throws. Larkspur: grain tests on every fact, a no-line-lost test, a re-sent-basket unit test (step 4). Elsewhere: an insurer asserting one row per claim per day before the reserving model reads it.

Snapshots: keeping history the source throws away

A snapshot is dbt’s Type 2 table (Part 4): each run compares the source’s current rows with the latest version of each key and closes a changed row (dbt_valid_to) as it inserts the new one. The timestamp strategy trusts an updated_at column; check compares listed columns.

What breaks without it. If the source overwrites in place, yesterday’s value exists nowhere else, and a snapshot can’t be rebuilt later (Gotcha). Larkspur: the catalogue lands a full extract daily, so dim_product is rebuilt from raw as a model; the promotions tool overwrites edits, so dim_promotion reads a snapshot. Elsewhere: a telecom’s tariff table, overwritten when a plan’s price changes.

Docs, packages, macros

dbt docs generate turns the graph, column descriptions and types into a static site. Packages are other people’s projects (dbt_utils 1.4.1 here). Macros are Jinja functions: use them where dialects differ, not to generate SQL nobody can read. What breaks without them. Analysts ask in chat what a column means; every team rewrites the same grain test. Larkspur: the column descriptions Farah’s analysts read, and local_date(), which lets one model file run on Trino and DuckDB (step 6). Elsewhere: a bank feeding its data catalogue from dbt’s docs artefacts.

dbt v1 and v2

dbt v2, formerly the dbt Fusion engine, is a rewrite in Rust, generally available since 14 September 2026. It ships as two distributions: dbt, proprietary under the dbt Product Licensing Agreement (adds SQL comprehension, linting and LSP support), and dbt-oss, Apache 2.0. Adapters are built in: GA for Snowflake, BigQuery, Databricks, Redshift and DuckDB (CLI only), Spark (beta, CLI only), ClickHouse (private beta). Trino isn’t listed. v2 parses strictly: a key v1 silently ignores is an error. dbt Core v1 stays Apache 2.0 and supported (1.12 has active support to July 2027).

So this series runs dbt-core 1.12 with dbt-trino for the lakehouse and shows dbt-oss with DuckDB alongside (step 6). dbt-trino 1.11.0’s changelog names dbt-core 1.11, and its pin (>=1.11.2,<2.0) allows 1.12: it ran this whole project on 1.12.6 without a problem. If v2 gains a GA Trino adapter, that changes.

The company changed too: Fivetran and dbt Labs announced their merger on 13 October 2025 and completed it on 1 June 2026, “initially operating as Fivetran + dbt Labs”. dbt Labs wrote that day that dbt Core remains Apache 2.0.

Under the Hood

What dbt build does

1. Parse. Read every .sql and .yml, render the Jinja far enough to find ref, source and config, and write the graph to target/manifest.json.
2. Compile. Render each node’s SQL for the target (profiles.yml): ref('stg_stores') becomes lakehouse_pipeline.silver.stg_stores.
3. Execute in graph order, threads at a time. In dbt build, a model’s unit tests run before it, its data tests after it, and a failure skips everything downstream, so a broken grain never reaches the marts people read. dbt run then dbt test builds everything first: use build.

A table on Iceberg through Trino

dbt-trino’s default for table is on_table_exists: rename: create a temporary table, rename the live table to a backup, rename the new one in, drop the backup. That drop is the weak step here: Trino drops Iceberg tables with purgeRequested=true, and Polaris refuses a purge unless the catalog allows it. The project sets on_table_exists: replace, so each build is one CREATE OR REPLACE TABLE: a single Iceberg commit on the same table, which keeps its snapshot history, so earlier builds stay readable by time travel (step 5).

Views are Iceberg views in Polaris. Snapshots MERGE each run’s changes through a staging table they create and drop, and that drop needs purging, so the lab sets polaris.config.drop-with-purge.enabled (what that costs: Gotcha).

Hands-On

The code is in the every-record-counts folder of the companion-workspaces repository (tag every-record-counts/part-17): the project in dbt/larkspur/, a DuckDB slice of it in dbt/larkspur_subset/, the driver in scripts/part17_dbt.py. Only core and Trino run. The commands, in step order:

make up-core && make up-trino
make dbt-install
uv run python -m scripts.part17_dbt setup
uv run python -m scripts.part17_dbt land
uv run python -m scripts.part17_dbt dbt build
uv run python -m scripts.part17_dbt replay --every 7
uv run python -m scripts.part17_dbt history
uv run python -m scripts.part17_dbt replay
uv run python -m scripts.part17_dbt history
uv run python -m scripts.part17_dbt dbt build
uv run python -m scripts.part17_dbt unit-demo
uv run python -m scripts.part17_dbt check
uv run python -m scripts.part17_dbt builds
uv run python -m scripts.part17_dbt dbt docs generate
make dbt-oss-install
uv run python -m scripts.part17_dbt dbt parse --target local
uv run python -m scripts.part17_dbt v2 parse --target local
uv run python -m scripts.part17_dbt strict
uv run python -m scripts.part17_dbt extract && uv run python -m scripts.part17_dbt subset-v2 debug
uv run python -m scripts.part17_dbt extract v1.5.0 && uv run python -m scripts.part17_dbt subset-v2 build
uv run python -m scripts.part17_dbt subset-v2 show --inline 'select version() as duckdb_version'
uv run python -m scripts.part17_dbt subset build
uv run python -m scripts.part17_dbt timing

Step 1: who dbt runs as, and what it reads

dbt connects to Trino through a new catalog file, lakehouse_pipeline: Polaris as Part 9’s pipeline principal, with vended storage credentials and no S3 key. setup grants what dbt needs beyond writing tables and sets the purge property from Under the Hood:

pipeline principal: + catalog role transformer on silver, gold:
  NAMESPACE_READ_PROPERTIES, TABLE_READ_PROPERTIES, TABLE_WRITE_PROPERTIES, VIEW_CREATE, VIEW_DROP, VIEW_LIST, VIEW_READ_PROPERTIES, VIEW_WRITE_PROPERTIES
catalog larkspur: polaris.config.drop-with-purge.enabled = true

The sources are raw tables plus Part 15’s checked silver.supplier_deliveries: Part 16’s CDC replicas of orders and refunds, and ten more that land copies from the raw zone in 9 s, including all 2,338,812 till events, re-sends and all. The profile, dbt/larkspur/profiles.yml (trimmed: host, port, user, http_scheme):

larkspur:
  target: lake
  outputs:
    lake:
      type: trino                    # the dbt-trino adapter
      method: none                   # no authentication in the lab; the catalog carries the identity
      database: lakehouse_pipeline   # in dbt-trino, the Trino catalog
      schema: silver                 # default namespace; models name theirs (a small macro stops "silver_gold")
      timezone: UTC                  # the session's zone; without it, the laptop's (Gotcha)
      threads: 2                     # models built in parallel (two: the lab Trino has a 2.4 GB heap)

Step 2: the models

dbt_project.yml gives each layer its namespace and materialisation, and every model +on_table_exists: replace. The re-send fix, models/staging/stg_pos_sale_lines.sql (comment and one column trimmed):

with first_upload as (
    select event_id
    from (
        select event_id,
               row_number() over (partition by basket_id order by uploaded_at, event_id) as upload_rank
        from {{ source('raw', 'pos_uploads') }}
    ) ranked
    where upload_rank = 1
)
select 'pos'                      as channel,
       b.basket_id                as transaction_id,
       l.line_no, b.store_id, b.currency, b.event_ts,
       b.tenders[1]."type"        as tender_type,
       l.sku, l.qty, l.unit_price_minor, l.discount_minor, l.promo_id,
       l.qty * l.unit_price_minor - l.discount_minor as amount_minor
from {{ source('raw', 'pos_uploads') }} b
join first_upload f on f.event_id = b.event_id
cross join unnest(b.lines) as l (line_no, sku, qty, unit_price_minor, discount_minor, promo_id)

Ranking whole events, nested lines and all, ran past the lab Trino’s 739 MB per-node query memory limit. The other models translate Parts 4 and 5 into Trino SQL, query for query. Two changes: surrogate keys are dbt_utils.generate_surrogate_key hashes, the same on every rebuild (Part 18’s incremental facts need that), and what Part 4 did with UPDATEs (placeholder products, the delivery milestone) a rebuilt model computes in one pass. A first dbt build creates everything; its snapshots, having seen a single morning, are rebuilt in step 3.

Step 3: snapshots, every morning or every week

snapshots/snapshots.yml (descriptions, comments and the product snapshot trimmed; it uses strategy: check over six attributes with updated_at: extracted_at):

snapshots:
  - name: snp_promotion
    relation: ref('stg_promotions_on_morning')
    config:
      unique_key: promo_id
      strategy: timestamp
      updated_at: updated_at
      dbt_valid_to_current: "timestamp '9999-12-31 00:00:00.000000 UTC'"

stg_promotions_on_morning is what the tool shows at 06:00 UTC on var('as_of') (the generator kept the history, so the lab can play any morning back). replay runs dbt build on both snapshots and their views once per morning, 1 June to 4 October, through dbtRunner:

126 runs, every 1 day(s) from 2026-06-01: 243 s (1.9 s per run)
  first run creates the tables: snp_product 1,867 rows, snp_promotion 1
  later runs that merged anything: 90 of 125 (245 product and 77 promotion rows)
snapshot         rows   keys  closed
snp_promotion      63     48      15
snp_product     2,056  2,000      56
snapshot vs extracts       2,056 vs 2,056 versions: 0 only in the snapshot, 0 only in the source
snapshot vs admin history  63 vs 63 versions: 0 only in the snapshot, 0 only in the source

Daily runs rebuild both histories exactly: Part 4’s 2,056 product versions and the tool’s 63 promotion versions. Weekly (--every 7, 18 runs), they didn’t:

snapshot vs extracts       2,054 vs 2,056 versions: 171 only in the snapshot, 173 only in the source
snapshot vs admin history  62 vs 63 versions: 0 only in the snapshot, 1 only in the source
never seen: PR-0045 v1, in force 2026-07-01 16:00:00.000000 UTC to 2026-07-05 18:00:00.000000 UTC, priced 1,908 sale lines

PR-0045 was created at 20% and edited to 25% four days later, between two weekly runs: version 1 exists nowhere, and its 1,908 sale lines would join to no promotion. Two product versions vanished the same way; 171 are dated by the run that noticed them, up to six days late. A snapshot only knows what was there when it looked.

Step 4: tests, in the build

Grain tests sit next to the models in models/marts/_marts.yml (excerpt, description trimmed):

  - name: fct_sales_line
    data_tests:
      - dbt_utils.unique_combination_of_columns:
          arguments:
            combination_of_columns: [transaction_id, line_no]

Every fact has one, and relationships tests tie the sales fact’s keys to their dimensions. fct_stock_daily’s grain test groups by a 64-bit hash of its keys (tests/generic/unique_combination_hashed.sql): two 32-character keys on 11.7 million rows ran out of memory. Singular tests check that no sale line or penny is lost between int_sales_lines and the fact, and that each product’s versions tile time. The unit test feeds stg_pos_sale_lines three events, two of them one basket (models/staging/_staging.yml, description and the fixture’s rows trimmed):

unit_tests:
  - name: a_resent_basket_counts_once
    model: stg_pos_sale_lines
    given:
      - input: source('raw', 'pos_uploads')
        format: sql
    expect:
      rows:
        - {transaction_id: B1, line_no: 1, sku: SKU-00001, amount_minor: 300, has_loyalty_card: false}
        - {transaction_id: B2, line_no: 1, sku: SKU-00002, amount_minor: 450, has_loyalty_card: true}

dbt build (lines trimmed):

18:03:38  Found 33 models, 30 data tests, 2 snapshots, 13 sources, 637 macros, 2 unit tests
18:03:39  17 of 67 PASS stg_pos_sale_lines::a_resent_basket_counts_once .................. [PASS in 0.27s]
18:04:40  Finished running 2 snapshots, 18 table models, 30 data tests, 2 unit tests, 15 view models in 0 hours 1 minutes and 2.17 seconds (62.17s).
18:04:40  Done. PASS=67 WARN=0 ERROR=0 SKIP=0 NO-OP=0 REUSED=0 TOTAL=67

A minute, mostly the three tables of 6.7 and 11.7 million rows (15–18 s each). unit-demo deletes the join first_upload line and runs the unit test (output trimmed):

18:04:43  1 of 1 FAIL 1 stg_pos_sale_lines::a_resent_basket_counts_once .................. [FAIL 1 in 0.44s]
actual differs from expected:
@@ ,transaction_id,line_no,sku      ,amount_minor,has_loyalty_card
   ,B1            ,1      ,SKU-00001,300         ,False
+++,B1            ,1      ,SKU-00001,300         ,false
   ,B2            ,1      ,SKU-00002,450         ,True

The +++ row is basket B1 again: the defect behind £118,076.98 of the 0.7%, caught in under half a second on three fixture rows.

Step 5: the same numbers as Part 5

check counts every table, compares the product and promotion version each sale line points at with Part 5’s DuckDB star, and runs sql/part17/check.sql as the analyst (dimension rows and the fill-rate line trimmed):

table                   dbt on Trino  Part 5 star
fct_sales_line             6,658,738    6,658,738  same
fct_basket                 2,506,704    2,506,704  same
fct_refund                    63,780       63,780  same
fct_stock_daily           11,659,200   11,659,200  same
fct_stock_adjustment          41,278       41,278  same
fct_po_line                   24,657       24,657  same
...
sale lines per (product version, promotion version): 7,040 groups, same
UK net revenue, Q3, GBP        29,790,674.06   manifest 29,790,674.06   exact
stock available, as reported        0.910509   manifest 0.910509
stock available, at close           0.885037   manifest 0.885037

£29,790,674.06, Part 5’s corrected UK net revenue to the penny, and stock at 91.1% as reported and 88.5% at close. builds lists gold.fct_sales_line’s Iceberg snapshots, one per build (eight today, most by test runs); FOR VERSION AS OF the first still returns 6,658,738 rows. dbt docs generate then took 4 s.

Step 6: the slice on dbt v2

First, the whole project under dbt-oss 2.0.8:

$ uv run python -m scripts.part17_dbt v2 parse --target local
[error] [SerializationError (dbt1013)]: Invalid model definition `larkspur.+on_table_exists`: Unrecognized key `larkspur.+on_table_exists`. Custom keys must go under `+meta`.
  --> dbt_project.yml:18:23

v2 has no Trino adapter, so a Trino setting is an unknown key, and unknown keys are errors; dbt-core 1.12 parsed the same project against the same target silently. strict shows why that silence matters, on a temporary copy of the slice with one British spelling added:

v1 parse, with `+materialised: view`     exit 0  no error, no warning
v2 parse, with `+materialised: view`     exit 1  [error] [SerializationError (dbt1013)]: Invalid model definition `larkspur_subset.+materialised`: Unrecognized
v2 parse --no-partial-parse              exit 0  no error, no warning
v2 run -m dim_date                       exit 0  no error, no warning

v1 builds tables while the author thinks they asked for views. The last two lines go the other way: per the upgrade guide, -m should error and --no-partial-parse should warn (dbt1700); 2.0.8 did neither. A Trino project can’t be half-moved to v2 today.

So dbt/larkspur_subset/ symlinks the eight portable models and builds them on data/lab/part17/extract.duckdb, three raw tables copied from the lake: v2’s built-in DuckDB driver can’t load httpfs or iceberg without a separately installed (dbc) driver. The first extract, written by DuckDB 2.0:

IO Error: Trying to read a database file with version number 999, but we can only read versions between 64 and 68.

dbt-oss bundles DuckDB v1.5.4 (select version() via dbt show; undocumented), which can’t open 2.0 files. Written with STORAGE_VERSION 'v1.5.0', the file works, and both engines build the slice with its six data tests and unit test. timing, three runs each:

(Medians; v1 parsed the full 33-model project in 2.23 s.) At this size most of v1’s time is fixed cost (0.75 s before any work); the queries take milliseconds in both. Big projects are where v2’s speed is claimed to matter; here, strict parsing is what you can count on.

The whole sequence peaked at 4.1 GiB of containers (Trino at its 3 GB limit) on a MacBook Pro M3 Pro with a 9.8 GB Docker VM; tests/test_part17.py replays it all and asserts the counts, versions and money above.

The Landscape

As of 28 October 2026, from each vendor’s docs:

Tips & Gotchas

Gotcha — the session time zone came from the laptop. Without timezone in the profile, the Trino client sends the machine’s zone. The first replay ran on an IST laptop, and CAST(snapshot_date AS timestamp(6) with time zone) dated all 2,056 product versions 18:30 UTC the day before. Any cast between dates and zoned timestamps depends on it: set timezone: UTC.

Gotcha — a snapshot table is a source, not a model. dbt can rebuild every model from raw, never a snapshot: what it recorded exists nowhere else. dbt’s --full-refresh leaves snapshots alone; the danger is a manual DROP to “start clean” (the lab’s replay does it only because it holds the history), which with purging on deletes the files too. Restrict who can drop them, and keep Iceberg snapshots long enough to undo a bad run.

Gotcha — dbt-core, dbt and dbt-oss all install a command called dbt. The docs say v2’s two distributions can’t share a Python environment; the lab keeps dbt-oss in .venv-dbt2.

Gotcha — purge on drop is catalog-wide. On Trino and Polaris, dbt snapshots need polaris.config.drop-with-purge.enabled (their staging-table drop is a purge), and then any principal with TABLE_DROP deletes a table’s files for good. Contain it with privileges: dbt’s principal gets TABLE_DROP on silver and gold only, the CDC writer has its own principal for raw, and no human role can drop snapshot tables. The lab’s single pipeline principal holds all three namespaces.

Tip — unit tests belong in development and CI. Their inputs are fixed, so running them in the production job proves nothing new; dbt’s docs recommend excluding them there (--exclude-resource-type unit_test).

Troubleshooting

Design Drill

The question: “A pharmaceutical company’s analytics team runs 900 SQL scripts from cron on one warehouse: sales by territory, distributor stock, batch-release lead times, and four reports for a regulator. Nobody knows the dependency order; when a script runs long, the next ones read half-built tables. Plan the migration.”

Clarify first

• Which outputs have consumers? (Query logs answer this, not the script folder.)
• Which regulatory reports must be reproducible as filed?
• Is the warehouse staying, or is this also an engine move? (One at a time.)
• Who owns models afterwards: a platform team or the analysts who wrote the scripts?

The shape of a strong answer

• Inventory first: the tables each script reads and writes (query history, or parsing the SQL) give the real graph; outputs nobody queried in 90 days are retired, not migrated.
• dbt Core 1.12 on the warehouse’s adapter (v2 if its adapter is GA); one repository, review on every change, CI building changed models and their children; dev and prod targets in separate schemas (Part 18).
• The half-built tables go away twice over: ref() orders the runs, so nothing reads a model before it’s built, and each table is replaced in one atomic swap (or a whole schema is built aside and switched in, blue-green), so readers see the old version or the new one, never half.
• Sources for tables no script creates, one staging model per source, shared logic in intermediate models.
• Migrate subgraph by subgraph, most-used reports first; each model runs beside its script until a parity test (counts, key sets, sums per day and territory, EXCEPT both ways) shows zero differences for an agreed period.
• Grain, not_null and relationships tests on every mart from day one; regulatory reports from tagged releases, with the manifest and run results archived per filing.
• A snapshot on day one for every source that overwrites history (price lists, territory assignments).
• An orchestrator calls dbt build (Part 19); cron goes with the last script.

Trade-offs to say out loud

• One script per model is fast and keeps the duplication; shift first, refactor behind parity tests.
• Full-table models are simple and slow; big facts go incremental after parity (Part 18).
• Running both side by side doubles warehouse cost for weeks: budget for it.

Common weak answers

• Translating 900 files mechanically, with no inventory.
• No parity testing: “the dbt version looks right”.
• Forgetting overwritten source history and the regulator’s need to reproduce a past filing.

Further Reading

1. dbt Labs, “Add snapshots to your DAG” (strategies, hard_deletes, dbt_valid_to_current). https://docs.getdbt.com/docs/build/snapshots
2. dbt Labs, “Unit tests”. https://docs.getdbt.com/docs/build/unit-tests
3. dbt Labs, “Starburst/Trino configurations” (on_table_exists, incremental strategies, contracts, snapshots on Iceberg). https://docs.getdbt.com/reference/resource-configs/trino-configs
4. dbt Labs, “Upgrading to dbt v2” (strict parsing, removed features). https://docs.getdbt.com/docs/dbt-versions/dbt-upgrade/upgrading-to-v2
5. Ralph Kimball and Margy Ross, The Data Warehouse Toolkit, 3rd edition (Wiley, 2013), chapter 5 on slowly changing dimensions.

Story Closing

Farah asked again. Rohan showed Farah a commit, the manifest that build wrote, and the sales fact’s Iceberg snapshots. “The Q3 number is that commit, built on that day. Change the code and the tests run before anything you read changes.”

Of the five promises from Part 2, this part strengthens correctness: Season 1’s corrected numbers now come from reviewed code with its tests in the build, not from a script on one laptop.

“Now look at the build log,” Ingrid said. “A minute, mostly rewriting 18 million rows that didn’t change to add one day. And a late till upload still has to land in the right day.”

In Part 18, the big facts go incremental, a local CI target builds only what a change touches, and dev and prod stop sharing one namespace.

This is Part 17 of “Every Record Counts: Data Engineering for Backend Engineers” — Season 2: Pipelines in Production.