Trino
4 items
Series Posts
Part 13: "Engines at Scale" — Trino, Cloud Warehouses and Real-Time OLAP
Three hundred people open the stores dashboard at 9 a.m. Trino, federation with PostgreSQL, ClickHouse, cloud warehouse billing, real-time OLAP, caches and materialised views, measured under load.
Part 17: "Transformations as Code" — dbt Fundamentals
Rohan wants to put the star schema's SQL in a repo and a cron job. Ingrid wants it treated like the order service: modules, a dependency graph, tests, review. Models, sources and refs, materialisations, snapshots, data and unit tests, docs, and dbt v1 vs v2. Built with dbt 1.12 and dbt-trino on Iceberg, it reproduces Part 5's corrected numbers to the penny; dbt-oss 2.0 then runs a slice of it on DuckDB.
Part 18: "dbt at Scale" — Incremental Models, CI and Environments
Rohan wants a high-water mark on event time to stop rebuilding the sales fact every night. Ingrid makes Rohan replay two weeks of till uploads first. Incremental strategies and how each treats late data, microbatch and its lookback, full-refresh policy, dev and prod targets with their own principals, state-based slim CI with deferral, contracts and model versions. On dbt 1.12 and Trino, the naive model loses 7,146 sale lines; microbatch with a two-day lookback matches a full rebuild row for row.
Part 24: "The Bill Always Comes" — Cost and Performance Engineering
Finance asks what the platform will cost per month and per report, and Rohan reaches for a cluster size. Ingrid meters the platform instead: bytes stored, bytes scanned, CPU and requests, priced with list prices as inputs. The biggest lines turn out to be one dashboard tile, a nightly run rereading history, and storage no table owns. Three fixes, each measured before and after, plus FinOps unit economics and the FOCUS billing schema.