"Platform, Not Pipelines" — Data Platform Engineering and Organisation

Skim Card

One thing to remember: a data platform is the checks, environments and ownership that let any team change data safely; the pipelines are what runs on it.

• Gate on what readers read: dbt’s tests passed a refund-dating change; reconciling the built report against the board pack caught +£41,975.38 in Q3.
• Compare at the reader’s grain: over all history that change moved UK revenue by −£32.81, yet all 378 country-days differed.
• One gate, three places: the same Make targets run in git hooks, on a laptop and in a GitHub Actions workflow (its first run passed).
• Run CI clean before trusting it: a fresh container found tests that silently needed Docker or the 7 GB raw zone.
• Environments share data, not copies: dev builds as its own principal and defers to production; promotion is a deploy, then the sign-off.
• Declare infrastructure and review the plan: OpenTofu flagged a hand-made GRANT as drift. Polaris has no released provider yet.
• Data mesh is an operating model, not a stack: take owners, contracts and a self-serve platform; decentralise only where domains can staff it.

Use …	When …
One central data team building the pipelines	A handful of sources; one team can know every number
A platform team plus domain-owned data products	Many producing teams, and the central team has become the queue
Data mesh, decentralised end to end	Domains have their own data engineers and the platform is mature

Verified against: OpenTofu 1.13.2 with the cyrilgdn/postgresql provider 1.27.0, pre-commit 4.7.0, ruff 0.17.3, datacontract-cli 1.2.8, dbt-core 1.12.6 + dbt-trino 1.11.0, Trino 484, actionlint 1.7.12, GitHub Actions (actions/checkout v7.0.1, astral-sh/setup-uv v10.3.3, opentofu/setup-opentofu v2.0.2) · vendor docs · 31 October 2026

Jump to: Concepts · Hands-On · Landscape · Design Drill

Story Opening

The loyalty team wanted its points and offers in the lakehouse. Rohan had the plan by lunch: a schema in PostgreSQL, a CDC connector, staging models, tests, a contract, a dashboard. “Two weeks. I’ve done it thirty times.”

“Thirty-three,” Ingrid said. “Merchandising asks next month, then stores, then forty more teams. You can’t be the pipeline for all of them, and you can’t be the CI either: Farah’s question last week was who runs the checks when someone else edits a model. Build the road, not the car.”

The Quick Map

Concepts, with use cases

Of Part 2’s five promises, this part strengthens consistency: every change, from any team, is held to the same checks and must leave the published numbers where finance signed them.

A platform, not a team that builds pipelines

A data platform is a set of self-service products other teams use without filing a ticket: storage and a catalog, engines, a way to declare sources, models and contracts, CI gates, environments, observability and access. Platform engineers call it a paved road: the supported way is also the easiest one.

What breaks without it. The central team becomes the queue, and each pipeline is a one-off nobody else can change. Larkspur: the loyalty domain gets a module (its schema, roles and grants), the repository’s make check, a contract template and Part 18’s dev namespace, not two weeks of Rohan. Elsewhere: a ride-hailing company lets every team register an event schema and get a governed table and a freshness monitor from it.

CI for data: gates in order of cost

CI for data runs every change through gates ordered by cost: static checks in seconds (lint, dbt parse, contract diffs), tests that need no services in minutes, then a build of the change on real data in an isolated environment, its data tests, and a reconciliation smoke test: the figures someone signed, recomputed from what the change built and compared with production at the grain readers use. After the deploy, Part 25’s sign-off and Part 22’s SLOs watch production.

What breaks without the last gates. Unit tests never see data; dbt’s data tests check shape (unique, not null), not meaning. Larkspur: a “tidy-up” that dates refunds by purchase passes everything except the reconciliation (Step 3). Elsewhere: a payroll provider runs last month’s payroll through every change and diffs each payslip.

Environments and promotion over shared data

An analytics environment is where a change’s outputs live and the identity that writes them; the inputs are shared, because copying a lake per developer is neither cheap nor current. Three patterns: a namespace and principal per environment that defers to production for anything unchanged (Part 18: dev, written only by dbt_dev through Part 23’s OPA policy); a branch per table, published by fast-forward (Part 20’s Write-Audit-Publish); and zero-copy clones in warehouses. Promotion is then either a code promotion (merge, then Part 18’s deploy rebuilds what changed in production) or a data promotion (fast-forward an audited snapshot).

Iceberg branches as whole environments have two limits today: a branch belongs to one table, so a change to fifteen models is fifteen branches and fifteen separate publishes; and Trino 484 still can’t create or write branches (trinodb/trino PR #31135 was open on 31 October 2026), so dbt on Trino can’t build into one. Catalog-level branching (Project Nessie 0.109, Apache 2.0) versions many tables in one commit.

What breaks without environments. Either developers test in production, or on a stale copy that passes while production fails. Larkspur: every CI build lands in an emptied dev. There is no long-lived staging copy: a per-change dev that reads production’s inputs tests the same data a staging copy would, without a third environment to keep in sync. Elsewhere: a bank’s model-risk team validates quarter-end changes on a clone taken that morning.

Infrastructure as code

Infrastructure as code (IaC) declares the desired state (schemas, roles, buckets, catalogs, grants) in files; the tool plans the difference against the real system and its state (its record of what it manages), a person reviews the plan, then the tool applies it. Modules package a pattern, which makes them the platform’s kit. Drift is the real system changed by hand; the next plan shows it.

Two tools dominate: OpenTofu 1.13.2 (MPL 2.0, the open-source fork) and Terraform 1.17.1 (Business Source License 1.1; its LICENSE names IBM as licensor since 11 March 2026), so Terraform is source-available, not open source.

What breaks without it. Grants nobody can explain, and environments that differ in ways nobody wrote down. Larkspur: source domains get their PostgreSQL kit from a module (Step 5). The catalog doesn’t: Apache’s terraform-provider-polaris repository says it “does not yet contain a usable provider” and has no release, so Polaris principals and grants stay Part 9 and Part 18’s idempotent setup scripts, which can’t plan or detect drift. Elsewhere: a hospital group rebuilds a region’s analytics stack from code after an outage.

Data products and data mesh, as they stand

A data product is a published interface with an owner, a contract and a monitored promise (Part 31). Data mesh (Zhamak Dehghani: article 2019, principles 2020, book 2022) organises them around four principles: domain-oriented decentralised ownership, data as a product, a self-serve data platform, and federated computational governance (global rules the platform enforces automatically).

Where it stands (judgement, not a survey). The parts adopted widely are owners per dataset, contracts (ODCS; Bitol’s Open Data Product Standard reached v1.1.0 in September 2026, Apache 2.0, LF AI & Data), self-serve platforms and policy as code. Full decentralisation, each domain building its own pipelines, is rarer: it needs data engineers in every domain and a platform mature enough to make the right way the easy way. “Computational” is the operative word: a governance committee is not a mesh; contract-ci and an OPA rule are.

What breaks if you copy the slogan. A reorganisation renames teams “domains” and nothing else changes. Larkspur: a hybrid: domains own their source contracts, finance owns the net revenue definition Farah signs, the platform owns the gates. Elsewhere: a retail bank keeps central reporting while card and mortgage domains publish their own products.

Team topologies, ownership and on-call

Team Topologies (Skelton and Pais) names four team types: stream-aligned (a flow of business work), platform (an internal product for the others), enabling (helps others over obstacles) and complicated-subsystem (deep specialist work); and three ways to interact: collaboration, X-as-a-Service and facilitation. At Larkspur: Rohan’s team is the platform team, offered as a service; the stores, online, loyalty and merchandising teams are stream-aligned; Ingrid and the reviewers form an enabling team; the people who own Part 21’s reconciliation are a complicated-subsystem team.

Ownership means one named team per dataset, written where tools read it (a contract’s team, the catalog), and on-call for data routes by cause: a producer’s contract break pages the producer, a platform outage the platform, a stale or wrong product its owner, who holds the SLO (Part 22). What breaks without it: “the data team” owns everything, so nobody does. Elsewhere: an airline’s operations domain carries the pager for on-time performance, not the BI team.

Under the Hood

The path of a change

Each gate runs the same Make target wherever it runs; only the services around it differ. The lake gate stays local because it needs the lab (Trino, Polaris, SeaweedFS and production’s tables); a CI job could run it against a shared CI environment.

What the smoke test compares

scripts/part34_platform.py smoke runs as dbt_dev, the identity CI builds with (it reads everything and writes only lakehouse_dev), and writes nothing. For each revenue report version the slim build put in dev, it full-joins the report with production’s on (country, business date) and counts the days that differ; for a full build it also sums Q3 for GB and compares it with the board pack’s uk_net_revenue_q3_minor. If the stock or supplier facts were rebuilt, it runs Part 17’s checks with dev substituted where it holds a table.

Windowed builds. The microbatch facts build only make dbt-ci’s event-time window (2 October 00:00 to 5 October 00:00 UTC), so a report rebuilt on them is complete only for business days entirely inside the window in all three time zones: 3 and 4 October. The smoke test compares those days, not the quarter; a test asserts the days follow the Makefile’s window. A comment appended to fct_sales_line and fct_stock_daily built 15 nodes; each report version compared 6 country-days, 0 different, and stock and supplier figures matched the board pack.

A change that should move a signed number is not a CI failure to override: it’s a restatement, which goes through Part 21’s process and Farah’s approval.

How a plan finds drift

tofu plan first refreshes: the provider reads each managed object from the system (for a schema grant, PostgreSQL’s catalog) and compares it with the configuration and with the state, the file that maps each resource address to its real object; apply writes the state. Two consequences. The state holds every attribute it manages, secrets included (a postgresql_role’s password attribute is stored there; password_wo is the documented way not to; the provider block’s own connection password is not written to state), so encrypt it (OpenTofu can encrypt state and plan files with AES-GCM) and keep it remote. And two people applying at once overwrite each other’s state, so remote state needs a lock: OpenTofu’s S3 backend can lock with a lock file in the bucket (use_lockfile, conditional writes) or a DynamoDB table.

Hands-On

The code is in the every-record-counts folder of the companion-workspaces repository (tag every-record-counts/part-34); the workflow is at the repository root, .github/workflows/every-record-counts.yml. The lake gate needs make up-trino (core, Trino, OPA; containers peaked at 5.32 GiB during the full make ci, Trino at its 3 GiB limit); OpenTofu needs only core, plus tofu on the PATH (1.13.2 here). New: pre-commit 4.7.0 in the dev dependencies.

make check                                    # the fast gate, no services needed
uv run --project every-record-counts pre-commit install -c every-record-counts/.pre-commit-config.yaml   # repo root
make up-trino && make check-lake              # the lake gate: slim build in dev + reconciliation smoke
uv run python -m scripts.part34_platform inject    # the Step 3 defect; `restore` undoes it
make infra-plan infra-apply infra-destroy     # Step 5

Step 1: one gate, wherever it runs

test-host:  ## the tests that need no lab service, the same set on any machine (Part 34)
  LAB_HOST_ONLY=1 uv run pytest -q

check: lint contract-ci dbt-parse test-host  ## the fast gate: lint, contracts, dbt parse, host tests (Part 34)

check-lake:  ## the lake gate (make up-trino): slim dbt build in an emptied dev, then the reconciliation smoke test (Part 34)
  uv run python -m scripts.part34_platform lake-check

Most of the lab’s tests skip themselves when a service is down, so on a laptop with the lab up, pytest would run a different set than on a CI runner. LAB_HOST_ONLY=1 makes the service-dependent skips the same everywhere: a hook in tests/conftest.py skips every test whose skipif names a service, and keeps those that only need a host tool (make dbt-install, make contracts-install). On the laptop, make check took 2 min 19 s: ruff, four contracts against origin/main, dbt parse, then 79 passed, 226 skipped in 131.12s (nine more pass here than on a runner, because this laptop has the raw zone and dbt-oss).

Step 2: on every commit

```yaml title=“.pre-commit-config.yaml (excerpt; name lines and four hooks cut)” minimum_pre_commit_version: “4.4.0” # language: unsupported (formerly system) default_install_hook_types: [pre-commit, pre-push] default_stages: [pre-commit]

repos:

• repo: https://github.com/astral-sh/ruff-pre-commit rev: 25bcf95026b96e2dd3c0aec3090c3d6ef6ca2e6c # frozen: v0.17.3 (keep in step with ruff in uv.lock) hooks:
  • id: ruff-check files: ^every-record-counts/
• repo: local hooks:
  • id: contract-ci entry: make -s -C every-record-counts contract-ci CONTRACT_BASE=HEAD language: unsupported files: ^every-record-counts/contracts/ pass_filenames: false
  • id: check-lake entry: make -s -C every-record-counts check-lake language: unsupported files: ^every-record-counts/(dbt|governance)/ pass_filenames: false stages: [pre-push]

The config lives in the folder, not the monorepo root, so `pre-commit install -c` writes its path into both git hooks. `pre-commit autoupdate --freeze` pinned ruff's hook repository to a commit, the same lesson as the workflow's actions below. Commit hooks take seconds; the host tests and the lake gate run on push. In a scratch clone, a "tidy-up" renaming `business_date` in the net revenue contract (Part 33) without a new version:

```text
$ git commit -am "net revenue contract: business_date is report_date"
data contracts (lint, no breaking change without a major version).......................Failed
- hook id: contract-ci
FAIL contracts/net_revenue.odcs.yaml: breaking change in version 1.0.0
       ERROR Removed schema.net_revenue.properties.business_date

(Skipped hooks, base: HEAD, the three ok lines and make’s error line trimmed.) As version: 2.0.0 the same commit passes, which is Part 20’s rule: a breaking change ships as a new major version, and the consumers sign off in review.

Step 3: the change dbt’s tests pass

The incident’s refund-dating defect (Part 4), reintroduced as a refactor: report version 1 dates refunds by purchase_date_key (“refunds sit with their sale”). git commit passed (dbt parse is all it runs for a model). What git push would run, without pushing:

$ pre-commit run -c every-record-counts/.pre-commit-config.yaml --hook-stage pre-push --from-ref HEAD~1 --to-ref HEAD
host tests (no lab services)...................................................................Passed
lake gate (make up-trino) - slim dbt build in dev + reconciliation smoke.......................Failed
state:modified+ selects 2 of 69 nodes: {'model': 1, 'test': 1, 'unit_test': 0}
22:58:23  Done. PASS=2 WARN=0 ERROR=0 SKIP=0 NO-OP=0 REUSED=0 TOTAL=2
dev holds: rpt_daily_net_revenue
rpt_daily_net_revenue: 378 country-days, all dates: 378 differ from production
    GB  -32.81
    IN  -1,245.49
    SG  +388.55
    Q3 GB net revenue 29,832,649.44, board pack 29,790,674.06
reconciliation smoke: FAIL rpt_daily_net_revenue

(The hook id, the models: line, dbt’s other log lines and make’s error line trimmed.) dbt built the model and its grain test passed: the shape is right. The meaning moved: refunds changed days, so every one of 378 country-days differs, Q3 moves by +£41,975.38, and over all history GB moves only −£32.81: an all-time total with any tolerance would have passed it. After restore, the change Rohan meant (a comment saying why refunds keep their own date) built the same two nodes and printed 378 country-days, all dates: 0 differ from production and Q3 GB net revenue 29,790,674.06, board pack 29,790,674.06: host tests 65 s, lake gate 10 s.

Promotion is Part 18’s: merge, deploy rebuilds state:modified+ in production, then Part 25’s sign-off recomputes the board pack read-only (today: 8 of 8 passed in 2.8 s).

And Farah’s question from Part 33, who runs the agent evaluation when someone edits a description in Cube? Not these gates, honestly: a cube/ change triggers only the host tests on push (Part 33’s model-free checks among them); its Cube tests need make up-cube and its trials a local model, so they belong in a scheduled or pre-release evaluation the semantic layer’s owners run, not built here. Also on the platform backlog: Part 33’s contract roles bind only the MCP server, until they are pushed down into the engine.

Step 4: the same gate on GitHub

```yaml title=“.github/workflows/every-record-counts.yml (the name: line and comment lines cut)” on: push: branches: [main] paths: - “every-record-counts/” - “.github/workflows/every-record-counts.yml” pull_request: paths: - “every-record-counts/” - “.github/workflows/every-record-counts.yml”

permissions: contents: read

concurrency: group: every-record-counts-${{ github.ref }} cancel-in-progress: true

jobs: check: runs-on: ubuntu-24.04-arm # the OS and architecture its steps were validated on; ubuntu-latest moves to 26.04 timeout-minutes: 30 defaults: run: working-directory: every-record-counts env: UV_LOCKED: “1” CONTRACT_BASE: ${{ github.event_name == ‘push’ && github.event.before || ‘auto’ }} steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: fetch-depth: 0 persist-credentials: false - uses: astral-sh/setup-uv@68559e99cc807c1eb7e9238fd43a15bba460eec2 # v10.3.3 with: version: “0.13.6” working-directory: every-record-counts enable-cache: true - uses: opentofu/setup-opentofu@a1320f892987e89d278cc92dc5adc984fb93aca4 # v2.0.2 with: tofu_version: “1.13.2” tofu_wrapper: false - name: Install dbt and datacontract-cli run: make dbt-install contracts-install - name: Lint, contracts, dbt parse, host tests run: make check - name: OpenTofu fmt and validate run: make infra-validate

The `paths` filter keeps the monorepo's other folders from triggering it, and `branches: [main]` keeps tag pushes out. Actions are pinned to commit SHAs (a tag can be moved), the token is read-only, `UV_LOCKED` fails on a stale lock instead of re-locking (datacontract-cli's own environment pins only its top-level version, so it is not fully locked), and `contract-ci` compares against the pushed range's start, or the merge base on a pull request; after a force-push removes that commit, it falls back to the merge base.

The workflow's first GitHub run, on the push of 31 October, [passed](https://github.com/phoenixtb/companion-workspaces/actions/runs/41184988946) in 2 min 45 s (`79 passed, 238 skipped`). It runs on `ubuntu-24.04-arm` (arm64 hosted runners are available to public repositories), not `ubuntu-latest`, which GitHub moves to Ubuntu 26.04 between 19 October and 19 November 2026: the label matches the OS and architecture its steps were validated on. actionlint 1.7.12 reported 0 errors (its shellcheck rule was off: not installed). Then its steps ran in a clean Ubuntu 24.04 container, no lab, no `data/`, with the same uv and OpenTofu versions; it took three runs to pass. The first, on amd64 under Rosetta, failed at collection: a Part 20 test module's skip condition called `docker` at import, and a runner may have none. (Rosetta also lacks the AVX instructions Polars needs, so the later runs used arm64, as the workflow now does.) The second found a Part 16 and a Part 27 test reading the 7 GB raw zone without the `lab_manifest` skip (and a wrong assumption in this part's own test). All now skip or pass cleanly; the third run passed: install 22 s, `make check` 1 min 39 s (`70 passed, 235 skipped`), `make infra-validate` 3 s.

### Step 5: the source-domain kit as code

```hcl title="infra/postgres/modules/source_domain/main.tf (excerpt)"
resource "postgresql_role" "reader" {
  name  = "${var.domain}_reader" # granted to the platform's CDC and extract users; reads only
  login = false
}

resource "postgresql_schema" "this" {
  name  = var.domain
  owner = postgresql_role.owner.name
}

resource "postgresql_grant" "usage" {
  database    = "larkspur"
  schema      = postgresql_schema.this.name
  role        = postgresql_role.reader.name
  object_type = "schema"
  privileges  = ["USAGE"]
}

The module also creates the owner role, SELECT on today’s tables and default privileges for tomorrow’s; main.tf pins the provider (cyrilgdn/postgresql 1.27.0, MPL 2.0) and calls the module for loyalty (owner team crm) and promotions (merchandising). The roles are NOLOGIN, so no password reaches the state. make infra-plan showed Plan: 12 to add; after infra-apply, someone “fixes” a ticket by hand:

$ docker compose exec -T postgres psql -U larkspur -d larkspur -c "GRANT CREATE ON SCHEMA loyalty TO loyalty_reader"
GRANT
$ make infra-plan
  # module.loyalty.postgresql_grant.usage will be updated in-place
  ~ resource "postgresql_grant" "usage" {
        id                = "loyalty_reader_larkspur_loyalty_schema"
      ~ privileges        = [
          - "CREATE",
            # (1 unchanged element hidden)
        ]
    }
Plan: 0 to add, 1 to change, 0 to destroy.

infra-apply took it back (1 changed); infra-destroy removed all 12. The state is a local file here; a team keeps it remote, locked and encrypted. tests/test_part34.py covers Steps 1–5: 7 passed in 27 s with Trino up.

The Landscape

Names as of 31 October 2026, from vendor pages; unverified cells say so.

Tips & Gotchas

Gotcha — a skipped check is a passed check in disguise. SKIP=check-lake git push works, and pytest’s “skipped” is green. Count skips in CI output; the lab’s lake gate fails, not skips, when Trino is down: keep it that way.

Gotcha — pin actions to a SHA, then keep the SHA current. A pinned action never updates itself; a tool such as Dependabot has to propose new SHAs. GitHub now also offers workflow dependency lockfiles (public preview, changelog of 14 October 2026): .github/workflows/actions.lock, made by the gh actions-lock extension; not used here.

Tip — slim CI needs production’s manifest. A fresh clone has no state/prod, and dbt stops: “Got a state selector method, but no comparison manifest”. Publish the manifest from every deploy as an artifact and download it in CI.

Gotcha — two owners for one object. If a setup script and OpenTofu both manage a role, each “fixes” the other’s changes forever. Move an object into IaC with tofu import and delete it from the script in the same change.

Troubleshooting

Design Drill

The question: “You are the first data platform hire at an online used-car marketplace: 300 engineers in 30 product teams, 60 PostgreSQL databases, a Snowflake account with 400 dbt models written by three analytics engineers, Airflow on one VM nobody owns, and dealers billed per active listing while two dashboards disagree on how many there were. What do you do in your first 90 days?”

Clarify first

• Which numbers cost money when wrong (active listings behind dealer bills, ads attribution), and who signs them?
• What already works, and what fails most: the top ten data products by consumer, with incident history?
• Headcount and budget: am I a team of one for a year?
• Constraints: GDPR, car-finance regulation, the cloud and contracts already signed?
• How do the 30 teams ship services today: CI, IaC, on-call culture?

The shape of a strong answer

• Days 1–30, listen and stabilise: inventory sources, models, dashboards and owners; back up and codify the Airflow VM and Snowflake roles (import into OpenTofu, not rebuild); make active listings the first product, with a written definition, an owner on the dealer side and a reconciliation to the listings database.
• A paved road v0 by day 45: a repository template with make check (lint, dbt parse, contract diff), slim CI per pull request into a zero-copy clone or a CI schema with deferral, and the listings reconciliation as the lake gate. Fast gate under three minutes or people route around it.
• Contracts where it hurts: the three source teams whose changes broke that number get contracts checked in their CI (Part 20), with CODEOWNERS making consumers reviewers.
• Environments as identities: dev, CI and prod roles in IaC; nobody’s personal user writes production.
• Ownership and on-call by day 90: every tier-1 dataset has an owning team in the catalog; SLOs on active listings and two others; pages routed by cause; the platform on-call covers Airflow and Snowflake.
• Measure the platform as a product: lead time to onboard a source, share of models behind CI, incidents caught before readers.
• A hiring plan: two platform engineers, and analytics engineers placed in the two busiest domains (enabling, then stepping back).
• Not in 90 days: no warehouse migration, no “data mesh” reorganisation, no tool shopping.

Trade-offs to say out loud

• Gates slow merges; a gate people bypass is worse than none. Keep the fast one fast and the slow one targeted.
• Central ownership gets the first number right sooner; domain ownership scales. Start central for tier 1, hand over as domains staff up.
• Buying (dbt platform CI, a managed orchestrator) saves months of a team of one; owning keeps cost and portability.

Common weak answers

• Picking tools first, or proposing a rewrite to a lakehouse.
• Building the pipelines personally, which makes the new hire the queue.
• “We’ll do data mesh” with no platform and no domain data engineers.

Further Reading

1. Zhamak Dehghani, “Data Mesh Principles and Logical Architecture” (martinfowler.com, 3 December 2020). https://martinfowler.com/articles/data-mesh-principles.html
2. Matthew Skelton and Manuel Pais, Team Topologies (IT Revolution, 2019); key concepts: https://teamtopologies.com/key-concepts
3. OpenTofu documentation: state encryption, and the S3 backend’s locking. https://opentofu.org/docs/language/state/encryption/
4. GitHub Docs, “Secure use reference” (pinning actions to commit SHAs, token permissions). https://docs.github.com/en/actions/reference/security/secure-use
5. pre-commit documentation (hook stages, language: unsupported, SKIP). https://pre-commit.com/

Story Closing

The loyalty team onboarded itself in three days: make infra-apply from a reviewed pull request, a contract, models through make check-lake. Rohan reviewed one diff.

“And when the lake gate goes red at 2 a.m. on a deploy?” Farah asked.

“Then it pages whoever owns what broke,” Ingrid said, “and the quarter’s numbers stay where you signed them.” Part 35 puts the whole platform through a final design review, batch, streaming, serving and AI together, with the interview playbook, and Farah closes the quarter.

This is Part 34 of “Every Record Counts: Data Engineering for Backend Engineers” — Season 3: Real Time, Serving and AI.