Design a Chat Messaging System

Prompt

"Design a messaging service: one-to-one and group chats, messages delivered in real time when the recipient is online and stored for later if not. Users have several devices."

Clarifying questions

• Group size limit? Up to 500 members. No broadcast channels with millions.
• Message types? Text plus media attachments referenced by URL; media upload is a separate service.
• Delivery semantics? A sent message must eventually reach every device of every member, in the same order for everyone in that chat. Duplicates must be hidden from users.
• Read receipts and typing indicators? Delivered and read receipts yes. Typing indicators are best-effort and can be dropped.
• Presence (online/offline)? Yes, coarse: online, away, offline, with a lag of tens of seconds acceptable.
• History? Full history stored server-side; devices sync on login. Retention indefinite unless the user deletes.
• End-to-end encryption? Out of scope; assume transport encryption only.

Requirements

Functional

• Send a message to a chat; deliver to all online member devices in real time; persist for offline devices.
• Consistent per-chat ordering across all recipients and devices.
• Delivered and read receipts per message per member.
• Device sync: a new or returning device fetches missed messages.
• Presence.

Non-functional

• Message delivery latency under 500 ms end to end for online recipients.
• No message loss once the sender's client receives an acknowledgement.
• Horizontal scalability in connections and in storage.
• Availability of send and receive over strict presence accuracy.

Back-of-envelope

• Assume 500 million daily active users, 40 messages sent per user per day: 20 billion messages/day, ~230 K sends/s average, peak ~700 K/s.
• Group fan-out: assume average 4 recipients per message across one-to-one and group; delivery events are ~4x sends, ~2.8 M/s at peak. Each recipient may have 2 devices, so ~5.6 M device deliveries/s at peak. Fan-out, not ingest, is the load.
• Concurrent connections: assume 20% of DAU online at peak with 1.5 devices each: 150 million persistent connections. At 200 K connections per gateway host, 750 hosts, plan for 1,000.
• Storage: message row ~300 bytes with metadata; 20 B x 300 B = 6 TB/day, ~2.2 PB/year, kept indefinitely. Media is stored elsewhere and referenced.
• Receipts: at 4 recipients per message, delivered plus read is ~8 receipt events per message, about 160 billion receipt events/day, but they are tiny and can be aggregated per chat rather than stored one row each.
• Inbox fan-out storage: if each device keeps a per-device queue of undelivered message ids, that is bounded by offline time; a device offline for a week in busy groups might have 10 K entries, trivial.