Estimate Log Volume per Day

Problem

A backend service handles about 2 billion API requests per day across a fleet of a few hundred servers. Every request emits application log lines; every server also emits system-level logs. Estimate the volume of logs generated per day in bytes, the ingest rate the logging pipeline must sustain at peak, and the storage required for a 30-day retention policy.

• Distinguish raw bytes from compressed bytes, and say which one each downstream component cares about.
• Give ingest in MB/s (average and peak) and storage in TB.
• Identify which single assumption you would verify first.

Examples

Starting figures; the interviewer will confirm or replace them on request:

• Requests per day: 2 × 10^9 (roughly 23,000 QPS average, ~70,000 at a 3× peak).
• Log lines per request: around 5: an access log line, one or two lines from business logic, a downstream-call log, and an occasional warning. Services with verbose debug logging emit 20+; disciplined ones emit 2-3.
• Bytes per line: ~200 bytes for a structured JSON line with timestamp, request ID, severity, service name, and a message. Access logs with full URLs and user agents run 300-500 bytes.
• Fleet size: ~300 servers.
• System logs per host: kernel, sshd, container runtime, agent heartbeats: on the order of 10 MB per host per day, and up to 100 MB if you count metrics scrapes as logs.
• Compression: text logs compress 8-12× with gzip or zstd because timestamps, field names, and service names repeat constantly. 10× is the accepted round figure.
• Indexing: a searchable log index typically adds 30-100% of the compressed size, depending on how many fields are indexed.

Hints

1. Logs scale with requests, not with servers. Start from request count × lines × bytes; the per-host system logs are a rounding error at this scale, and proving that is part of the answer.
2. The pipeline (network, Kafka, collectors) sees raw bytes at peak; the storage system sees compressed bytes × retention. Keep the two separate.
3. Retention is a multiplication by days, and storage is where the surprisingly large number lives; 30 days of 2 TB/day is not small.