Design an Email Service

Prompt

"Design a web email service. Users send and receive mail, browse their inbox, and search their own messages. Think of the scale of a major consumer provider."

Clarifying questions

• Do we need to interoperate with other providers? Yes. Inbound and outbound mail must speak standard SMTP with the rest of the internet; internal delivery can use whatever we like.
• Attachments? Yes, up to 25 MB per message. Attachments dominate storage.
• Search: full text over body and attachments, or headers only? Full text over headers and body. Attachment text is a stretch goal.
• Labels or folders? Labels: a message can carry many labels; the inbox is just a label. This shapes the data model.
• Read receipts, threading, spam filtering? Threading yes (group by conversation). Spam classification is a black-box service we call; we do not design the classifier. No read receipts.
• How much storage per user? 15 GB quota. Most users use far less.
• Consistency expectations? A user who sends a message must see it in Sent immediately; a recipient can see a new message a few seconds after delivery. Two devices of the same user should converge quickly.

Requirements

Functional

• Send mail (to internal and external recipients) and receive mail from anywhere.
• List messages by label, sorted by time; open a message and its attachments.
• Search a user's own mail by keyword, sender, date range, label.
• Multi-device sync: a change on one device (read, archive, label) shows up on others.

Non-functional

• Durability is paramount: a lost email is the one thing users never forgive. Target no data loss short of losing multiple regions.
• Inbox load under 300 ms p99 for the first page; search under 1 s p99.
• Availability of read paths above write paths: users tolerate "sending is delayed" better than "cannot open inbox".
• Privacy: search indexes are per-user and never cross-user.

Back-of-envelope

• Assume 1 billion active users, 40 messages received and 10 sent per user per day.
• Messages per day: 1 B x 50 = 50 billion. Per second: 50 B / 86,400 = about 580 K messages/s average, peak ~1.5 M/s. Most inbound volume is spam and is rejected or quarantined early, so the stored rate is lower; still, plan the ingest tier for the full number.
• Message size: ~75 KB average once attachments are amortised (text-only messages are ~5 KB, but roughly 10% carry attachments averaging ~700 KB). Storage per day: 50 B x 75 KB = 3.75 PB/day. Per year: ~1.4 EB raw. Deduplicating identical attachments and newsletters, plus compression, realistically cuts this by 3-5x, but it is still hundreds of PB per year: this is a storage-first design.
• Read QPS: assume each user opens the inbox 10 times per day and reads 20 messages: 30 B reads/day, ~350 K/s average. Reads are cheaper than writes here because they hit the metadata store and only fetch bodies on open.
• Search index: text-only content is ~5 KB per message; an inverted index typically costs 20-40% of the text size, so ~1.5 KB per message, 50 B x 1.5 KB = 75 TB/day of index growth. That is per-user sharded but large.
• Bandwidth: outbound SMTP at 10 B messages/day x 75 KB is 750 TB/day, ~9 GB/s. Meaningful but not the constraint.