Design a Chat System
A chat system must deliver messages in real time, maintain message order, support presence indicators, and handle millions of concurrent connections. The challenge is coordinating a persistent bidirectional connection layer with a durable, ordered message store.
Design it yourself
Don't just read it — drag components onto a canvas and get Aria's interviewer review.
Requirements
Functional
- 1:1 messaging — send and receive messages between two users in real time
- Group messaging — support groups of up to 500 members
- Message persistence — users can scroll back through full history
- Online/offline presence — show whether a contact is currently online
- Read receipts — single tick (sent), double tick (delivered), blue tick (read)
- Media sharing — images, videos, and files (stretch goal)
Non-Functional
- 50B messages per day ≈ 580,000 messages/sec at peak
- 500M daily active users, each with ~10 open connections
- Message delivery latency < 100ms for online recipients
- Messages must be durable — never lost after the server acknowledges receipt
- 99.99% availability — users notice immediately if chat is down
Capacity Estimation
| Messages / day | 50B ≈ 580,000 / sec |
| DAU | 500M users |
| Concurrent connections | ~50M (10% of DAU active at once) |
| Storage per message | ~100 bytes (text) + metadata |
| Storage / day | 50B × 100B = 5 TB / day |
| Storage (5 years) | ~9 PB (with compression and tiering) |
High-Level Components
Client
Mobile or web client maintains a persistent WebSocket connection to a Chat Server. Falls back to long-polling for environments that block WebSockets.
Chat Server (WebSocket Nodes)
Stateful servers that hold open WebSocket connections. When a message arrives, it is written to Kafka and the server attempts immediate delivery to any online recipient connected to the same node. Horizontally scaled — each node handles ~100K concurrent connections.
Message Service
Persists messages to the Message Store. Assigns a monotonically increasing sequence ID per conversation. Publishes the message to Kafka for fan-out to recipient chat servers.
Message Queue (Kafka)
Decouples message ingestion from delivery. Each conversation maps to a Kafka partition — guaranteeing message ordering per conversation. Chat servers consume from Kafka to deliver messages to connected recipients.
Message Store (Cassandra)
Stores messages partitioned by conversation_id, ordered by sequence_id DESC. Cassandra's wide-column model is ideal: the hot read path is "last N messages in conversation X" — a single partition scan.
Presence Service
Tracks online/offline status. Clients send a heartbeat every 5 seconds. Status is stored in Redis with a 10-second TTL — if the heartbeat stops, the key expires and the user is considered offline. Presence updates are broadcast to interested parties via a pub/sub channel.
Push Notification Service
For offline users, the Message Service triggers push notifications via APNs (iOS) or FCM (Android). Notifications carry a snippet of the message and a badge count.
API Gateway
Handles authentication (JWT validation), WebSocket upgrade routing, and REST endpoints for history, user search, and media upload URLs.
Architecture Diagram
Deep Dives
WebSocket Connection Management
HTTP is request-response — unsuitable for real-time push. A WebSocket upgrades an HTTP connection to a full-duplex TCP channel that stays open. The server can push frames to the client at any time without the client polling.
Connection scaling: A single Linux process can hold ~1M concurrent TCP connections (with tuned file descriptor limits). In practice, target ~100K connections per chat server to leave headroom. 50M concurrent users → 500 chat server nodes.
Sticky sessions: A user's WebSocket must land on — and stay on — the same chat server for its lifetime. Use consistent hashing at the load balancer: hash(userId) → chat server. If a server dies, affected clients reconnect and rehash to a live server.
Heartbeat: Client sends a PING frame every 25 seconds. Server responds with PONG. If three consecutive pings go unanswered, the client reconnects. This prevents silent connection drops (e.g. NAT timeout on mobile networks).
Java — WebSocket endpoint with presence on open/close
// Spring Boot WebSocket — minimal chat endpoint
@ServerEndpoint("/ws/chat")
@Component
public class ChatEndpoint {
private static final Map<String, Session> SESSIONS = new ConcurrentHashMap<>();
@OnOpen
public void onOpen(Session session) {
String userId = extractUserId(session);
SESSIONS.put(userId, session);
presenceService.markOnline(userId);
}
@OnMessage
public void onMessage(String payload, Session session) {
ChatMessage msg = objectMapper.readValue(payload, ChatMessage.class);
messageService.send(msg); // persist + publish to Kafka
deliverIfOnline(msg.recipientId(), payload); // attempt immediate delivery
}
@OnClose
public void onClose(Session session) {
String userId = extractUserId(session);
SESSIONS.remove(userId);
presenceService.markOffline(userId);
}
void deliverIfOnline(String userId, String payload) {
Session s = SESSIONS.get(userId);
if (s != null && s.isOpen()) {
s.getAsyncRemote().sendText(payload);
}
}
}Message Ordering and the Sequence ID Problem
Messages must be displayed in the order they were sent. Two naive approaches fail at scale:
Client timestamp: Clocks across devices drift — a message "sent" at 12:00:00.001 on one device may have a lower timestamp than a message sent at 12:00:00.000 on another with a slow clock. Do not trust client clocks for ordering.
Database auto-increment: A single auto-increment primary key becomes a write bottleneck at 580K messages/sec.
Better: per-conversation sequence counter in Redis. Redis INCR is atomic and handles millions of ops/sec. Each conversation has a counter key. When a message arrives, INCR the counter → that number becomes the message's sequence_id. Monotonically increasing per conversation, no global bottleneck.
For Cassandra, use sequence_id as the clustering column (DESC) so the "last N messages" query is a prefix scan.
Cassandra — messages table with per-conversation ordering
-- Cassandra table design
CREATE TABLE messages (
conversation_id UUID,
sequence_id BIGINT, -- per-conversation counter from Redis INCR
sender_id UUID,
content TEXT,
type TEXT, -- text | image | video | file
sent_at TIMESTAMP,
delivered_at TIMESTAMP,
read_at TIMESTAMP,
PRIMARY KEY (conversation_id, sequence_id)
) WITH CLUSTERING ORDER BY (sequence_id DESC);
-- Query: last 50 messages in a conversation
SELECT * FROM messages
WHERE conversation_id = ?
ORDER BY sequence_id DESC
LIMIT 50;Group Message Fan-out
When a user sends a message to a group of 500 members, the message must be delivered to all 500. Two strategies:
Fan-out on write: The Message Service creates one message record per recipient at write time. Each recipient's inbox is pre-populated. Read path is simple — just read your own inbox. Write amplification: 1 send → 500 DB writes. Suitable for small groups.
Fan-out on read: Store one copy of the message. At read time, the client fetches the group's message list. No write amplification. But reading is more expensive and group messages arrive with higher latency.
Hybrid (used by WhatsApp/WeChat): Fan-out on write for small groups (< 200 members). Fan-out on read for very large groups or broadcast channels. The threshold is tunable.
Kafka for delivery: The Message Service publishes once to a group topic. Chat Servers subscribe and fan out to their locally connected members. Kafka handles the distribution — not the Message Service.
Presence at Scale
Naive approach: store online status in a SQL table. Updating 500M rows/day is untenable.
Redis TTL approach: On WebSocket connect, SET user:{id}:online 1 EX 10 (10-second TTL). Client sends heartbeat every 5 seconds, which resets the TTL. On disconnect (or missed heartbeats), the key expires naturally — no explicit offline write needed. Checking presence: EXISTS user:{id}:online → O(1).
Broadcasting presence changes: When user A comes online, their contacts should see the green dot. Maintain a pub/sub channel per user (user:{id}:presence). Interested parties (chat servers holding connections of A's contacts) subscribe. Presence events are broadcast — not polled.
Privacy: Users should be able to hide their last-seen time. Store a privacy setting per user and gate presence reads accordingly.
Java — Redis TTL-based presence with pub/sub broadcast
// Presence Service — heartbeat handler
public void heartbeat(String userId) {
// Reset TTL on every heartbeat
redis.setex("presence:" + userId, 10, "online");
}
public boolean isOnline(String userId) {
return redis.exists("presence:" + userId);
}
public void broadcastOnline(String userId) {
// Notify subscribers (chat servers holding connections of this user's contacts)
String event = objectMapper.writeValueAsString(Map.of(
"userId", userId,
"status", "online",
"timestamp", Instant.now().getEpochSecond()
));
redis.publish("presence:" + userId, event);
}Offline Message Delivery
When the recipient is offline, the message is persisted but cannot be pushed via WebSocket. Two delivery mechanisms:
Push notification: Message Service triggers APNs / FCM with a notification payload (sender name, message snippet, badge count). When the user taps the notification, the app opens and fetches recent messages via REST.
Pull on reconnect: When a client reconnects via WebSocket, it sends its last_seen_sequence_id for each active conversation. The Message Service returns all messages with sequence_id > last_seen. This guarantees no messages are missed even if push notifications are blocked or delayed.
Key Trade-offs
WebSocket vs long-polling
Full-duplex, low overhead, sub-100ms delivery. Long-polling adds ~1 extra HTTP round-trip per message and is more complex to scale.
Fan-out on write vs fan-out on read for groups
Fan-out on write is fast to read but amplifies writes 500x for large groups. Fan-out on read is cheaper to write but slower to read. Hybrid uses write for small groups, read for large ones.
Per-conversation sequence ID (Redis INCR) vs global sequence
A global sequence is a single-point bottleneck at 580K msg/sec. Per-conversation counters distribute the load across all active conversations.
Cassandra vs MySQL for message store
Messages are append-only, partitioned by conversation, and accessed by conversation + time range. Cassandra's data model and horizontal scalability match perfectly. MySQL would require painful sharding.
Interview Tips
- 1Clarify whether 1:1 only or group chat is in scope — group fan-out is a completely different problem.
- 2Start with the WebSocket connection model. Interviewers expect you to know why polling does not work at this scale.
- 3The message ordering problem (don't trust client clocks) always impresses. Bring up per-conversation Redis counters.
- 4Mention the critical path: send → Kafka → persistence → delivery. Show you can identify what must be synchronous vs async.
- 5Bring up offline delivery — push notifications + pull on reconnect. This shows product thinking.
- 6For group chat, the fan-out write amplification problem is a classic deep-dive. Know the hybrid strategy.
Discussion
Discussion
Sign in to join the discussion.