"Time Is a Lie" — Streaming Fundamentals

Skim Card

One thing to remember: in a stream, when it happened and when you saw it are different clocks; correct answers need event time, a watermark and a declared lateness policy.

• Window by event time: grouped by arrival, a van’s 23 late readings blurred into one window, and a real excursion went unquarantined.
• A watermark is a guess about completeness. A 2-minute one silently dropped 14 readings and lost the excursion’s first window.
• Declare a lateness policy for every output: drop late data, update the result within an allowed lateness, or divert it to a side output.
• Lateness costs: allowed lateness was right by 10:27, with 50 sink rows and twice the state; a 10-minute watermark was right at 10:35.
• Exactly-once needs a replayable source, state and offsets snapshotted together, and a sink that commits with the snapshot or upserts.
• Test lateness with per-event watermarks: replayed with Flink’s 200 ms default, the same query dropped nothing.

Use …	When …
Processing-time windows	The question is about the system (throughput, lag), never about the business event
Event time, short watermark, late data dropped	Late events can’t change the decision: live tiles, surge pricing
Event time plus allowed lateness or a side output	Late events must count: compliance, billing; the sink upserts
Batch over event time with a lookback	The decision can wait an hour or a day

Verified against: Apache Flink 2.2.1 (Java 17) with the Flink SQL Kafka connector 5.0.0-2.2, Apache Kafka 4.4.0, Python 3.13 · vendor docs · 30 October 2026

Jump to: Concepts · Hands-On · Landscape · Design Drill

Story Opening

The store manager’s request from the end of Season 2 became Rohan’s first streaming job. Chilled-goods vans report their compartment temperature every 30 seconds, and Larkspur’s rule says two 5-minute windows in a row averaging above 5.0 °C quarantine the load. Rohan wrote it like any consumer: poll Kafka, put each reading in the bucket for the current 5 minutes, average the bucket when the clock moves on.

A week later a Manchester store shelved yoghurt whose van had been above 5 °C for about ten minutes, peaking at 8.9 °C. The van had been unloading in an underground bay with no signal; the sensor kept its 23 readings and uploaded them in about two seconds when the van drove out. Rohan’s job put every one in the bucket for “now”, saw one warm window, not two, and quarantined nothing.

“Your job answered when did I hear about it,” Ingrid said. “The rule asks when did it happen. In a stream those are different clocks.”

The Quick Map

Concepts, with use cases

Of Part 2’s five promises, this part strengthens timeliness without giving up completeness: answers in minutes that still count every reading.

Bounded and unbounded data

Bounded data has an end: a day’s settlement file, a table as of last night. Unbounded data doesn’t: tills, clicks and sensors keep producing. Seasons 1 and 2 cut everything into business dates and declared a lateness window after which a day was final. A stream has no “after the file landed”: you answer while data still arrives.

What breaks if you ignore it. A job that waits for “all the data” never answers. Larkspur: the cold-chain readings never end, and the store needs the answer while the van is at the door. Elsewhere: a card issuer can’t wait for the end of the day to spot a stolen card.

Event time and processing time

Event time is when something happened, by the clock that saw it: the sensor’s event_ts. Processing time is when a system handles it. Ingestion time is when it entered your platform: the MQTT bridge’s ingest_ts. The gap between event and processing time, the skew, is seconds on a good day and unbounded on a bad one. Processing time is the right clock only for questions about the system itself, such as consumer lag.

What breaks without the distinction. Processing-time results depend on network weather and on when you replay: the same readings give different answers on different days. Larkspur: the van’s 10:13–10:24 readings arrived at 10:24:05, so a processing-time job averaged them with 10:20–10:25’s and found one warm window instead of two. Elsewhere: a fitness tracker syncs a whole run when the phone reconnects; steps per minute by sync time put the run into one minute.

Windows

A window cuts an unbounded stream into finite groups you can aggregate. Four shapes cover most needs:

Names collide: the Dataflow paper, Apache Beam and Flink’s DataStream API call the hopping shape “sliding”; Flink SQL calls it HOP.

What breaks without windows. An unbounded GROUP BY never closes: nothing is final, and state grows forever.

Watermarks

A watermark is a timestamp flowing through the stream that asserts “no event older than this is still coming”. Few sources can promise that, so most watermarks are heuristics. The common one is bounded out-of-orderness: the largest event time seen so far minus a bound you choose, here 2 minutes. When the watermark passes a window’s end, the window fires.

The Dataflow paper names the two ways a heuristic fails: “sometimes too fast”, so late data arrives behind it, and “sometimes too slow”, because one slow input holds back the whole pipeline.

What breaks without one. Event-time windows never know when to fire. Larkspur: a 2-minute bound covers the lab’s network jitter (13 readings out of order, at most 45 seconds late) and loses nothing until a van goes underground. Elsewhere: an ad exchange billing per minute needs a cut-off after which the minute’s line is final.

Allowed lateness and side outputs

Late data arrives after the watermark has passed its window’s end. Three honest options:

• Drop it. Cheap and simple; fine when a few lost events don’t change a decision.
• Update the result: keep the window’s state for an allowed lateness after the watermark passes, and fire again for each late event.
• Divert it to a side output, a second stream of late events for a batch correction or a human.

What breaks without a declared policy. The engine picks one: Flink’s default allowed lateness is zero, so late events vanish without an error. Larkspur: the compliance record needs every reading, so the cold-chain job updates (lateness 10 minutes) and diverts anything later to a table the compliance team reviews. Elsewhere: surge pricing for a ride-hailing app happily drops late GPS pings; the same company’s driver payouts must divert them, because a late trip still gets paid.

Triggers and refinements

A trigger decides when a window emits a result: when the watermark passes (the default), early on a timer, or on each late event. Each emission is a pane, and panes relate in one of three ways:

• Discarding: each pane carries only what’s new since the last one.
• Accumulating: each pane is the whole window so far; downstream overwrites the old value.
• Accumulating and retracting: each new pane follows a retraction of the previous one, so downstream aggregations can subtract it; Flink SQL’s -U/+U rows.

What breaks without it. An early estimate and a final value can’t come from one job. Larkspur: the store’s live tile updates every few seconds; the quarantine decision waits for the watermark. Elsewhere: an election-night broadcast shows running counts, then the certified result.

State and where it lives

Windows, joins and deduplication are stateful: the operator keeps, per key, what it has seen: a sum and a count for an average, every distinct value for COUNT(DISTINCT). State grows with keys × open windows × how long you keep them, so allowed lateness costs memory. Flink 2.2 keeps it on the JVM heap by default (HashMapStateBackend). When state outgrows the heap you choose a local RocksDB per task instead (state.backend.type: rocksdb); Flink doesn’t switch by itself. ForSt (experimental) can keep its files on remote storage.

What breaks without planning for it. State you never expire is a slow memory leak with a pager attached. Larkspur: vans × 5-minute windows is small; Part 29’s per-SKU stock-out detector isn’t. Elsewhere: distinct cards per device over 24 hours is most of a fraud feature’s cost.

Delivery semantics, end to end

At-most-once may lose inputs, at-least-once may repeat them, exactly-once means each affects the result once. Part 21 showed that producers’ re-sends need a business key. Inside a stream processor, exactly-once needs three more things:

1. a replayable source (a Kafka offset you can rewind to);
2. a consistent snapshot of all operator state and the source offsets it reflects, taken together;
3. a sink that commits with the snapshot or is idempotent (an upsert keyed by van and window).

Miss the first two and a restart loses or repeats input; miss the third and the state is right while the table downstream holds duplicates. Larkspur: a quarantine record counted twice blocks good stock; counted zero times it ships warm yoghurt. Elsewhere: usage-based billing for an API platform, where a replayed minute is an overcharge.

Lambda, kappa and the Dataflow model

The lambda architecture (Nathan Marz) runs a batch layer for correct results beside a speed layer for fast approximate ones, merged at query time, so every piece of logic is written twice. Jay Kreps’ 2014 essay “Questioning the Lambda Architecture” proposed kappa: one streaming codebase; to reprocess, replay the log into a new output. The Dataflow model (Akidau et al., VLDB 2015) describes what you compute, where in event time (windows), when in processing time (triggers) and how refinements relate, in semantics “general enough to subsume the standard batch, micro-batch, and streaming models”; the engine becomes a cost and latency choice.

Larkspur’s hybrid. Finance stays batch: it must match settlement files, and 06:00 is fine. Streaming adds answers worth something only in minutes; the side output feeding a batch correction is lambda in miniature, for a few late events. Elsewhere: a bank’s fraud scores and its end-of-day ledger read the same events on different clocks.

Under the Hood

Who moves the watermark

Flink generates the watermark at the source, per Kafka partition, and each operator takes the minimum over its inputs. The lab’s topic has one partition for all four vans, so the fleet’s newest reading sets it: when VAN-014’s burst arrived at 10:24:05, the other vans had pushed it to 10:21:58, past the 10:10 and 10:15 windows. That is “too fast”.

Give each van its own partition and VAN-014’s silent one would hold the watermark at 10:10:30 (10:12:30 minus 2 minutes) for eleven minutes, delaying every van: “too slow”. An idle timeout (scan.watermark.idle-timeout) lets the others advance, which makes the burst late again. No setting fixes both; the lateness policy decides.

What a window operator decides, per record

A record whose window’s end plus the allowed lateness is at or below the watermark goes to the side output; any other joins its window’s state (a sum and a count) and, if the window already fired, fires it again. Then the watermark moves to max(event_ts) − bound, windows whose end it reached fire, and windows past their end plus lateness are deleted: that is all of scripts/part26_time.py’s operator, written to Flink’s rules.

What a checkpoint captures

Flink injects barriers into the source streams (“barriers never overtake records”). Sources record their offsets at the barrier; an operator with barrier n from every input snapshots its state and passes it on. On failure Flink “selects the latest completed checkpoint”, restores every operator’s state and rewinds the sources to the recorded offsets. State and input match because they were captured at the same cut of the stream. Output is the remaining gap, closed by two-phase sinks: pre-commit at the barrier, commit when the checkpoint is confirmed complete. Iceberg’s Flink sink does exactly this (“Iceberg commit happened after successful Flink checkpoint in the notifyCheckpointComplete callback”), which Parts 28 and 29 use.

Hands-On

The code is in the every-record-counts folder of the companion-workspaces repository (tag every-record-counts/part-26). Steps 1 to 4 are host Python only: datagen/coldchain.py generates the readings, scripts/part26_time.py simulates the window operator. Step 5 starts Kafka and a Flink 2.2.1 session cluster. Peak container memory was 2.55 GiB, Kafka and Flink 1.54 GiB of it (core was running, not needed).

uv run python -m scripts.part26_time generate
uv run python -m scripts.part26_time windows
uv run python -m scripts.part26_time late
uv run python -m scripts.part26_time crash
make up-flink                                   # Kafka + Flink JobManager + TaskManager
uv run python -m scripts.part26_time kafka
uv run python -m scripts.part26_time flink      # runs flink/sql/part26/windows.sql, checks it against step 2

Step 1: the readings, in the order they arrive

Four vans, one reading every 30 seconds from 10:00 to 10:45 UTC (11:00 in Manchester), each with the sensor’s event_ts and the bridge’s ingest_ts, written in arrival order (trimmed: the middle of the burst):

360 readings from 4 vans, 10:00-10:45 UTC, written in arrival order to data/lab/part26/readings.jsonl
out of order by jitter: 13 (delays up to 45 s); buffered in VAN-014's dead zone: 23 (up to 665 s late)

Arrival order around 10:24 UTC (event_ts = sensor clock, ingest_ts = bridge clock):
van      seq  event_ts  ingest_ts     temp_c
VAN-023   48  10:23:58  10:24:00.937  3.5
VAN-014   27  10:13:00  10:24:05.000  3.3
VAN-014   28  10:13:30  10:24:05.100  3.6
…
VAN-014   49  10:24:00  10:24:07.200  6.4
VAN-011   49  10:24:07  10:24:10.181  2.9

The jitter is harmless; the 23 loading-bay readings, up to 665 seconds late, carry the whole excursion (peak 8.9 °C).

Step 2: one stream, five answers

windows computes VAN-014’s averages five ways; decided at is the arrival time when two consecutive flagged windows were first visible downstream.

VAN-014, 5-minute windows by start (UTC): average °C (readings); * = above 5.0 °C
strategy                            10:10         10:15         10:20         10:25         quarantine  decided at
processing time (arrival)           3.17  (6)     —             6.36* (24)    4.05  (10)    no          -
event time, all data                3.40  (10)    6.17* (10)    7.59* (10)    4.05  (10)    yes         -
watermark 2 min, late dropped       3.17  (6)     —             7.59* (10)    4.05  (10)    no          -
watermark 2 min + 10 min lateness   3.40  (10)    6.17* (10)    7.59* (10)    4.05  (10)    yes         10:27:02
watermark 10 min                    3.40  (10)    6.17* (10)    7.59* (10)    4.05  (10)    yes         10:35:02

Processing time is Rohan’s job: no 10:15 window, and the 10:20 window averages the 24 readings that arrived then. The 2-minute watermark closed the 10:10 and 10:15 windows before the burst arrived; the jitter inside the bound cost nothing, since every other van’s windows are exact (the tests check this). Allowed lateness and the 10-minute watermark both reach the truth, eight minutes apart.

Step 3: what lateness costs

strategy                             side output  late firings  rows to sink  open windows
watermark 2 min, late dropped                 14             0            35             8
watermark 2 min + 10 min lateness              0            14            50            16
watermark 10 min                               0             0            36            13

watermark 2 min: side output = VAN-014 readings 10:13:00-10:19:30, all arriving 10:24:05-10:24:06

The 14 dropped readings are exactly what a side output catches. Allowed lateness turns them into 14 late firings, one per reading (the 10:10 window fired at 10:17:09, then again at 10:24:05.000, .100, .200 …), so the sink gets 50 rows for 36 windows and must treat a repeat as an update. At its peak it holds twice as many open windows in state, 16 against 8, because each lives 10 minutes past its end. The 10-minute watermark sits between on state and answers last. Correctness, latency, cost: the Dataflow paper’s title says you balance them; nothing maximises all three.

Step 4: a crash in the middle of the burst

crash replays the stream (2-minute watermark, 10-minute lateness), checkpoints every minute of arrival time, and crashes 12 readings into the burst; the last checkpoint was just before it. Three ways to recover:

• at-most-once: offsets committed as readings are polled, window state only in memory;
• at-least-once: a hand-rolled consumer writing state to an external store per reading and committing offsets at each checkpoint (not Flink’s AT_LEAST_ONCE mode, which restores state and repeats only records that overtook a barrier);
• exactly-once state: state and offsets snapshotted together at each checkpoint, both restored.

final value per window = the last row the sink got for it (what an upsert sink holds)

recovery model               10:10         10:15         10:20  equals no-crash  rows to sink
at-most-once            3.40  (10)     7.55* (2)    7.59* (10)            False            49
at-least-once           3.50  (14)    6.02* (18)    7.59* (10)            False            62
exactly-once state      3.40  (10)    6.17* (10)    7.59* (10)             True            63
no crash                3.40  (10)    6.17* (10)    7.59* (10)                             50

exactly-once state: rows the sink got for VAN-014 10:10, in order: 3.17, 3.19, 3.24, 3.31, 3.40, 3.19, 3.24, 3.31, 3.40

At-most-once rebuilt the 10:15 window from the 2 readings after the crash. At-least-once re-read 13 readings onto state that already had them: 18 readings in a 10-reading window. Exactly-once state restored state and offset from the same checkpoint and every window’s last row matches, yet the sink received 63 rows, 13 more than without the crash: results emitted after the checkpoint came again, so the 10:10 value went back from 3.40 to 3.19 before recovering, and an alert on it could fire twice. An upsert sink ends right but shows the wobble; a transactional sink, committing only completed checkpoints, never shows it (requirement 3).

Step 5: the same windows in Flink SQL (preview)

Parts 28 and 29 teach Flink; this step only checks that a real engine agrees. make up-flink starts Kafka and a session cluster from flink:2.2.1-java17 plus the Kafka SQL connector 5.0.0-2.2. Flink 2.2.1, not 2.3: no released Kafka connector supports 2.3 yet. Java 17, not the series’ Java 25: Flink 2.2 supports 11, 17 (“recommended”) and 21 (experimental), and its serialization reflects into JDK internals, so each new JDK is porting work.

Settings go in FLINK_PROPERTIES, which the image writes into config.yaml (Flink 2.x dropped flink-conf.yaml); each key is explained in compose.yaml. kafka writes the readings to a one-partition topic; flink runs this script in the SQL client inside the JobManager (make flink-sql SQL=part26/windows.sql):

CREATE TABLE readings (
  van_id     STRING,
  sensor_id  STRING,
  seq        INT,
  temp_c     DECIMAL(4, 1),
  event_ts   TIMESTAMP(3),   -- the sensor's clock, UTC
  ingest_ts  TIMESTAMP(3),   -- the MQTT bridge's clock, UTC
  WATERMARK FOR event_ts AS event_ts - INTERVAL '2' MINUTE
) WITH (
  'connector' = 'kafka',
  'topic' = 'coldchain.readings',
  'properties.bootstrap.servers' = 'kafka:29092',
  'scan.startup.mode' = 'earliest-offset',
  'scan.bounded.mode' = 'latest-offset',        -- replay what is there, then finish
  'scan.watermark.emit.strategy' = 'on-event',  -- a watermark after every reading, not every 200 ms
  'format' = 'json'
);

-- The same topic and columns; only the watermark differs.
CREATE TABLE readings_wm10 (
  WATERMARK FOR event_ts AS event_ts - INTERVAL '10' MINUTE
) LIKE readings (OVERWRITING WATERMARKS);

SELECT van_id, window_start, COUNT(*) AS readings, CAST(AVG(temp_c) AS DECIMAL(4, 2)) AS avg_c
FROM TABLE(TUMBLE(TABLE readings, DESCRIPTOR(event_ts), INTERVAL '5' MINUTES))
WHERE van_id = 'VAN-014' AND window_start >= TIMESTAMP '2026-10-14 10:10:00' AND window_start < TIMESTAMP '2026-10-14 10:30:00'
GROUP BY van_id, window_start, window_end;

The watermark belongs to the table: WATERMARK FOR makes event_ts an event-time attribute with a 2-minute bound, and LIKE … (OVERWRITING WATERMARKS) copies the table with another. TUMBLE, like HOP, CUMULATE and SESSION, is a window table-valued function that adds window_start and window_end to each row. scan.bounded.mode ends the replay at the topic’s end, and the finished source moves the watermark to the end of time, so every window fires.

-- watermark 2 min, late dropped
+----+--------------------------------+-------------------------+----------------------+--------+
| op |                         van_id |            window_start |             readings |  avg_c |
+----+--------------------------------+-------------------------+----------------------+--------+
| +I |                        VAN-014 | 2026-10-14 10:10:00.000 |                    6 |   3.17 |
| +I |                        VAN-014 | 2026-10-14 10:20:00.000 |                   10 |   7.59 |
| +I |                        VAN-014 | 2026-10-14 10:25:00.000 |                   10 |   4.05 |
+----+--------------------------------+-------------------------+----------------------+--------+
same as the simulation's 'watermark 2 min, late dropped': True

Flink dropped the same 14 readings silently: a missing 10:15 row and a 6 where there should be 10. The second query (trimmed), over readings_wm10, returned step 2’s event-time row exactly: 3.40, 6.17, 7.59, 4.05 (same as the simulation's 'watermark 10 min': True). Window TVF aggregations emit each window once (+I); late updates as in step 3 are the DataStream API’s allowed lateness, in Part 29. tests/test_part26.py checks the numbers above, the Flink run included when Flink is up.

The Landscape

Current product names as of 30 October 2026. A cell that couldn’t be verified says so.

Tips & Gotchas

Gotcha — fast replays hide lateness. Flink SQL emits watermarks every 200 ms of wall-clock time by default (pipeline.auto-watermark-interval); the script’s 'scan.watermark.emit.strategy' = 'on-event' emits one per record. Replayed with the default (flink/sql/part26/replay_periodic.sql, three runs), the 2-minute watermark dropped nothing, presumably because the whole replay outran the first interval. Your backfill passes and production drops data: test lateness with per-event watermarks.

Gotcha — a window with no data emits nothing. Step 2’s dropped strategy reported no 10:15 window at all, and “no alert” looks like “no data” downstream. Alert on missing windows per device, not only on bad values.

Gotcha — device clocks drift. A sensor whose clock runs minutes slow files every reading in the wrong window, and no watermark can tell. Keep both clocks, as event_ts and ingest_ts do here, and alert when their gap per device moves.

Tip — size the bound from measured skew, lateness from the business. Plot ingest_ts − event_ts per source; let the bound cover ordinary jitter and handle outages with allowed lateness or a side output, not by delaying every result by the worst case.

Tip — the simple option is often right. If the decision can wait for the next hour, an hourly batch over event time with a lookback (Part 18’s microbatch) is a stream processor you already run.

Design Drill

The question: “We stream live football to up to 20 million concurrent viewers. Product wants a ‘watching now’ counter on screen, and the rights holders want unique viewers per minute for the match report. Design it.”

Clarify first

• What is a viewer: an account, a device or a playback session?
• What signal exists: player heartbeats (how often?), CDN logs, or both?
• How fresh and how exact: the counter within seconds, the report within hours, with what error?
• How late can heartbeats arrive (mobile networks, players reconnecting)?
• Do restreams and bots count? Is the global number the sum of regions?

The shape of a strong answer

• Define both metrics first, from player heartbeats, not CDN logs: watching now is distinct accounts with a heartbeat in the last minute, refreshed every few seconds; unique viewers per minute is distinct accounts per tumbling minute of event time.
• Size it: 20 M viewers at one heartbeat per 30 s is about 670,000 events a second. Partition the topic by viewer ID.
• Two numbers, two designs: the counter is approximate and early (in Flink SQL, CUMULATE with a 5-second step and 1-minute size, or a hopping window); the report is exact, from batch over the same events in the lake.
• Approximate with HyperLogLog (Part 6): 2^14 registers is about 12 KB per minute at 6 bits each, standard error near 0.8% (1.04/√m). Sketches merge across partitions and regions; distinct counts don’t add.
• Avoid the hot key: the match is one key. Build one sketch per partition per minute and merge them, or, for exact counts in Flink SQL, split-distinct aggregation (table.optimizer.distinct-agg.split.enabled).
• Watermark bound from measured heartbeat skew (tens of seconds), idle timeouts for quiet partitions, a few minutes of lateness for the counter, a side output of later heartbeats into the lake.
• Replays are harmless to a sketch: adding the same ID twice changes nothing, and late heartbeats merge into the stored minute’s sketch. The report is idempotent per (match, minute); the counter is labelled an estimate.

Trade-offs to say out loud

• Approximate and instant vs exact and late: two answers for two audiences.
• A longer watermark is steadier and slower; allowed lateness is quick but holds state for every open minute.
• Sketches merge cheaply but can’t remove a viewer: retractions need exact state.

Common weak answers

• COUNT(DISTINCT user_id) GROUP BY match, minute on processing time: one hot key, and a CDN hiccup becomes a dip, then a spike.
• Summing per-region distinct counts, double-counting viewers who move between them.
• Never defining “viewer”, so the number disagrees with the rights holder’s.

Further Reading

1. Akidau et al., “The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing”, PVLDB 8(12), 2015. https://www.vldb.org/pvldb/vol8/p1792-Akidau.pdf
2. Tyler Akidau, Slava Chernyak and Reuven Lax, Streaming Systems (O’Reilly, 2018): the book-length version of watermarks, triggers and exactly-once.
3. Apache Flink 2.2 documentation, “Stateful Stream Processing” (checkpoint barriers and recovery). https://nightlies.apache.org/flink/flink-docs-release-2.2/docs/concepts/stateful-stream-processing/
4. Apache Flink 2.2 documentation, “Windowing TVF” and “Time Attributes”. https://nightlies.apache.org/flink/flink-docs-release-2.2/docs/dev/table/sql/queries/window-tvf/
5. Jay Kreps, “Questioning the Lambda Architecture” (O’Reilly Radar, 2014). https://www.oreilly.com/ideas/questioning-the-lambda-architecture

Story Closing

Rohan rewrote the job on event time with a 2-minute watermark, 10 minutes of allowed lateness and a side output the compliance team reviews each morning. Replayed through it, the Manchester delivery is quarantined at 10:27, three minutes after the van left the bay.

“And readings later than ten minutes?” Farah asked.

“A table, with both timestamps, that the morning batch applies,” Ingrid said. “Late isn’t lost. It’s just late.”

Four vans on one partition was a lab. Larkspur’s real stream needs topics designed for analytics: keys, partitions, retention, and schemas that evolve without breaking jobs. In Part 27, Rohan builds the event backbone on Kafka, with a schema registry that says no.

This is Part 26 of “Every Record Counts: Data Engineering for Backend Engineers” — Season 3: Real Time, Serving and AI.