What it is
A queue holds items in the order they arrive (or by priority) so a producer and a consumer can run at different speeds without either blocking on the other. At small scale this is a language-level structure — a deque tracking events inside a trailing time window. At system scale it becomes an infrastructure component in its own right: a durable, partitioned log that absorbs bursts, lets consumers fall behind and catch up, and lets independent stages of a pipeline fail and retry without losing work. What changes between the two scales is what you must guarantee about ordering, durability, and delivery once a single in-memory list can't hold everything.
When it applies
- You need to answer "how many things happened in the trailing window" and old entries should drop off automatically as new ones arrive — a small in-memory queue does this without rescanning history each time.
- Two parts of a system produce and consume work at different, unpredictable rates, and a burst in one should not become a cascading failure in the other — stick a durable queue at the boundary so each side can be scaled and retried independently.
- Work needs to fan out to many consumers (many subscribers to one topic) or fan in from many producers to shared processing stages, and each stage needs its own failure and retry boundary.
- Strict global ordering across everything ever produced is something a single queue or partition can give you, but partitioning for throughput (the normal scaling move) only preserves ordering within a partition — don't promise cross-partition ordering a partitioned queue can't provide.
Tells in the problem
- "Hits/requests/events in the last N seconds" with a stream of arrivals and expiring old ones.
- "Producers publish, consumers subscribe," "at-least-once delivery," or "messages must not be lost" — durability and delivery-guarantee language, not just a data structure.
- A burst description with numbers ("a spike of N should be absorbed in seconds and drained over minutes") — the queue's job is explicitly to decouple enqueue rate from drain rate.
- Per-key or per-host ordering required, but not global ordering — a hint that the design needs many parallel queues (partitions, per-host lanes), not one.
- A pipeline description where each stage is separated by "queue," so a failure in one stage doesn't lose or duplicate work already accepted by the stage before it.
The classic mistake
At small scale, the common bug is treating a trailing window as a fixed-size buffer instead of expiring by time — counting hits by index or a rolling array size rather than checking each entry's actual timestamp against the current one, which breaks the moment calls arrive irregularly rather than one per unit time. At system scale, the parallel mistake is promising an ordering or delivery guarantee the chosen queue shape can't actually provide — claiming full global ordering across a topic that's partitioned for throughput, or treating "the message was enqueued" as equivalent to "the message was durably committed" when a producer's write hasn't yet been acknowledged by enough replicas to survive a machine loss. Say precisely what's ordered (per-key, not globally) and what's durable (acknowledged and replicated, not merely accepted) rather than letting the word "queue" imply guarantees you haven't built.
Questions to start with
- Design Hit Counter — the smallest version: a trailing time window over a queue of timestamps, with expiry driven by time rather than a fixed count.
- Design a Notification System — a queue at every pipeline stage boundary, sized explicitly to absorb a stated burst before providers' rate limits become the bottleneck.
- Design a Distributed Message Queue — the queue as the whole product: durability, per-key ordering, retention, and pull-based consumption with explicit position commits.
- Design a Web Crawler — a two-level priority-and-per-host queue structure, needed because a single priority queue can't satisfy both "crawl the most valuable URL next" and "never hit one host too often" at once.