What it is
Aggregation design questions ask you to build a system that ingests a firehose of events (clicks, metrics, searches) and answers a rollup question over a time window — a count, a rate, or a top-k list — fast, continuously, and at a volume where recomputing from raw events on every query is not an option. The core design decision is always the same shape: pre-aggregate as events arrive (streaming counters, sliding windows, sketches) rather than scanning raw history per query, and decide up front which of exactness, latency, and cost you're willing to trade against the others for this particular use case.
When it applies
- Events arrive continuously and at high volume, and the read side needs a summary (a count, a rank, a rate) over a recent window, not the raw events themselves.
- The volume or cardinality is too large to keep an exact answer cheaply (millions of distinct keys, billions of events per day) and an approximate structure or approach is acceptable, or explicitly called out as acceptable for one class of query while another (billing, alerting) demands exactness.
- Late-arriving or duplicate events are a stated concern, and the design must define a policy for them rather than assume a clean, ordered stream.
- If the system's job is instead to durably store and retrieve individual pieces of content on request (a file, a message, a video) rather than to summarize a stream of many small events, that's the storage pattern, not aggregation — the giveaway is whether the primary read is "give me this one thing" or "give me a rollup over many things."
Tells in the problem
- "How many X in the last M minutes" or "top K by count over the last hour/day" as the core query shape.
- Explicit freshness requirements ("under one minute of lag") paired with an explicit tolerance for approximation on some queries but not others (dashboards approximate, billing exact).
- Mentions of duplicate events, retries, or late/out-of-order arrival as things the design must account for, not edge cases to wave away.
- Very high cardinality of the thing being counted (hundreds of millions of distinct keys) — a hint that exact per-key counting isn't affordable and a probabilistic structure (count-min sketch, HyperLogLog) or a bounded top-k structure belongs in the design.
- A stated need for multiple time granularities or retention tiers (raw for days, downsampled for months, coarser for years).
The classic mistake
A design built only for the average case load glosses over the two things that actually break averages: duplicate/out-of-order events (which silently double-count or undercount if not deduplicated and windowed correctly) and the exactness-versus-approximation line, left implicit instead of stated. A design that never explains how it distinguishes a retried click from a second real click, or that promises exact top-100 rankings over hundreds of millions of distinct keys without acknowledging the cost of doing so exactly, looks complete on the whiteboard but collapses under the first follow-up question. State explicitly, before diagramming components: which queries in this system must be exact (and why — usually billing or alerting), which can tolerate approximation, and what happens to a duplicate or a late event under your chosen windowing strategy.
Questions to start with
- Design an Ad Click Event Aggregation System — the clearest exactness split in the set: approximate dashboards versus exact, deduplicated billing numbers from the same event stream.
- Design Top-K Heavy Hitters — introduces approximate ranking over an effectively unbounded key space, where exact counting per key is not affordable.
- Design a Metrics Monitoring and Alerting System — the largest scope: multiple retention tiers, and an alerting path that must not miss a sustained violation even while the rest of the system tolerates approximation.