Project Every Record Counts: Data Engineering for Backend Engineers
Part 22: "Know Before They Do" — Observability, Lineage and SLAs
Part 21's gate held the revenue report back, correctly, and the first to notice was Farah. Rohan makes the data page the team instead: freshness, completeness, volume and schema as SLOs with an error budget counted in days, OpenLineage from Airflow, dbt and Spark loaded into OpenMetadata to trace the sales and stock reports to their sources, OpenTelemetry for the runs, and a "the dashboard is wrong" runbook.